-- Logs begin at Wed 2022-10-05 13:31:38 UTC, end at Wed 2022-10-05 15:08:17 UTC. -- Oct 05 13:56:38.397434 dv-cinder-835878-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 05 13:56:38.404956 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Oct 5 13:56:38 2022] *** Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: nodename: dv-cinder-835878-4-iscsi Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: machine: x86_64 Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: clock source: unix Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: pcre jit disabled Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: detected number of CPU cores: 4 Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: current working directory: / Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: detected binary path: /usr/bin/uwsgi-core Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: your processes number limit is 47680 Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: your memory page size is 4096 bytes Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: detected max file descriptor number: 2048 Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: lock engine: pthread robust mutexes Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: thunder lock: enabled Oct 05 13:56:38.408439 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 05 13:56:38.409345 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 05 13:56:38.447442 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: Python main interpreter initialized at 0x5635d0eae9c0 Oct 05 13:56:38.447442 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: python threads support enabled Oct 05 13:56:38.447442 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: your server socket listen backlog is limited to 100 connections Oct 05 13:56:38.447442 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: your mercy for graceful operations on workers is 90 seconds Oct 05 13:56:38.447834 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: mapped 403077 bytes (393 KB) for 2 cores Oct 05 13:56:38.447834 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: *** Operational MODE: preforking *** Oct 05 13:56:38.448138 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: *** uWSGI is running in multiple interpreter mode *** Oct 05 13:56:38.448138 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: spawned uWSGI master process (pid: 81556) Oct 05 13:56:38.448355 dv-cinder-835878-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 05 13:56:38.449607 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: spawned uWSGI worker 1 (pid: 81570, cores: 1) Oct 05 13:56:38.449607 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: spawned uWSGI worker 2 (pid: 81571, cores: 1) Oct 05 13:56:38.449607 dv-cinder-835878-4-iscsi devstack@keystone.service[81556]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 05 13:56:42.645146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 13:56:42.645146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: from cryptography.utils import int_from_bytes Oct 05 13:56:42.646018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 13:56:42.646018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Oct 05 13:56:42.646439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 13:56:42.646439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: from cryptography.utils import int_from_bytes Oct 05 13:56:42.646544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 05 13:56:42.646544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: from cryptography.utils import int_from_bytes Oct 05 13:56:42.694531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 05 13:56:42.694724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 05 13:56:42.694798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 05 13:56:42.694874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] command line args: [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 05 13:56:42.694936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 05 13:56:42.695139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 05 13:56:42.695272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] command line args: [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 05 13:56:42.695345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ================================================================================ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 05 13:56:42.695411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 05 13:56:42.695554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.695685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ================================================================================ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 05 13:56:42.695780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_dir = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.695853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] admin_token = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.695922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.695986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_dir = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] config_source = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] debug = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] config_source = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] debug = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.696956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_config_append = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.697968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_config_append = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_dir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698302 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_dir = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.698934 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699191 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.699808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700698 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.700883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.701024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.701103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.701173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.701339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.701461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.701540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.701624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.701860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] publish_errors = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] publish_errors = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.702940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.703057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.703136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.703314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.703452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.703540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] standard_threads = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.703668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.703781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.703905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] standard_threads = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_journal = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_json = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_journal = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.704989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_stderr = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.705097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_json = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.705204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] use_syslog = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.705325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_stderr = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.705442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.705551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] use_syslog = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.705725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.705923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 05 13:56:42.706041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.706162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.706273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.706389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.706498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.706612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.706727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.706844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.706960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.707945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.708936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.709050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.709159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.709274 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.709385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.709506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.709667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.709800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.709911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.710996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.711113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.711222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.711336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.711445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.711562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.711673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.711781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.711889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712240 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712355 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.712924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.713933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.714935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.715045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.715159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.715271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.715378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.715486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.715670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.715782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.715892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.716898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.717993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718578 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.718926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.719935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.720947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.721056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.721165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.721273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.721382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.721490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.721598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.721784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.721896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.722890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.723901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.724902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.725933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.726997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.727115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.727233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.727233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.727451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.727561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.727670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.727779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.727888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728450 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728787 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.728895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.external = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.729924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.external = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.password = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.730872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] auth.token = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.password = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] auth.token = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.731995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.732106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.732215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.732335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.732447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.732556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.732667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.732776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.732886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.733911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.734940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735484 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.735823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.736028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.736028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.736556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.736556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.736790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.736902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.737011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.737205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.737319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.737439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.737551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.737675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.737788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.737898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738465 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.738915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.739947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.740133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.740247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.740358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.740467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.740580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.740689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.740807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.740918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.741973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742211 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.742999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.743110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.743220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.743331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.743447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.743566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.743677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.743791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.743902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.744911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745355 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.745942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.746962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747185 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.747973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.748982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.749092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.749201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.749311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.749422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.749540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.749680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.749793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.749903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750787 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.750904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751221 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.751984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.752976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.753086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.753201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.753310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.753420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.753537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.753667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.753782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.753904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.754931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.755935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.756942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.757055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.757164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.757274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.757383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.757534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.757682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.757798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.757909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758127 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758469 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.758908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.759940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.760937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.761054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.761166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.761276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.761385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.761494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.761604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.761833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.761950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.762957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763191 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763631 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.763976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.764984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.765103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.765215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.765325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.765435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.765545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.765708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.765834 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.765948 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.766947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.767847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.768345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.768465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.768575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.768695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.768808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.768918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.769936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.770952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.771951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.772958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.773069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.773178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.773296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.773416 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.773537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.773667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.773790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.773900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.774923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.775936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776266 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.776938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.777048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.777169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.777281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.777281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.777488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.777597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.777784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.777908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.778021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.778131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81570) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 05 13:56:42.778241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.778241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.778241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.778241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.778241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.778753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 05 13:56:42.778753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81571) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 05 13:56:42.806122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.806359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.806577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.806735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.806940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.807166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.814783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.815442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.843548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.843787 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.845399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.845593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.897101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.897356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.945932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.946204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.947516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:42.947646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.106814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.106996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.107895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.130752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.130880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131581 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.131913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.132037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.132187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.143007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.143324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.155697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.155928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.156112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.156232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.156364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.156443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.156536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.156621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.156774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.156899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.157026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81571) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.157109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.157171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.157392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81570) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 05 13:56:43.173577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.173813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 05 13:56:43.174155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.174278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.174415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 05 13:56:43.174515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 05 13:56:43.174761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.174887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.175021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 05 13:56:43.175099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.175231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.175311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.175467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.175590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.175725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.175955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.185107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.185384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.185506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.185669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.185752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.185932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.186056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.186194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.192976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.193103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.195875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.196002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.196131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.196214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.196426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.196551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.196689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.196775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.196981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.197106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.197242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.197335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.197474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.197561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.197791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.197913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.198070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.198154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.198307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.198432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.198538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.198625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.220856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.220983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.222390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.222516 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.222651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.222788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.227331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.227456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.227595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.227675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.227870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.227994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.234188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.234313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.234446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.234525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.240370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.240497 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.240635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.240714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.240929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.241073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.241212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.241292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.241451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.241542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.241764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.241888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.242024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.242104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.242272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.242397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.257418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.257686 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.257785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.258099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.261513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.262009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.264668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.264956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.265189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.265507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.269114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.269763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.272342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.272863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.275527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.275860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.276140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.276262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.276391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.276511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.276655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.276820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.276945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.277082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.277356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.277677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.292554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.292812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.293105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.293332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.293432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.293675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.293936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.294200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.300173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.301066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.301689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.302633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.303231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.303502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.304159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.304432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.306179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.306458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.306734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.307011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.307134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.307367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.307490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.307627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.307994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.308396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.318443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.318726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.319005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.319282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.319572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.319715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.319854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.319933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.320122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.320246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.320407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.320766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.321041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.321335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.335406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.336674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.338300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.339595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.341128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.342481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.342808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.343919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.344392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.344682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.345076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.345376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.345675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.346097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.351339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.352490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.352778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.353967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.355634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.356022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.356300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.356571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.356889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.357222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.357512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.357765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.364687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.364980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.365252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.366000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.366300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.366579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.372044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81570) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.372325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.372596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.372866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.373142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.373400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81571) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 05 13:56:43.373524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.373716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.373800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.373952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.374321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.374411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.374549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.374642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.374758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.374838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.374911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.374984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.375100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.375176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.375257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.375467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81570) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.375607 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.375777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.376067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.376334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.376583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.376861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81571) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 05 13:56:43.408929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5635d0eae9c0 pid: 81570 (default app) Oct 05 13:56:43.410573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5635d0eae9c0 pid: 81571 (default app) Oct 05 13:56:43.412249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aade30-e8e0-42f4-b587-03c3c7543d39 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:43.412526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aade30-e8e0-42f4-b587-03c3c7543d39 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:43.412787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aade30-e8e0-42f4-b587-03c3c7543d39 None None] PATH_INFO: `/v3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:43.417056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Wed Oct 5 13:56:43 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:56:50.225834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1716e-f6c7-4f75-ac20-5ff694b62662 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:50.226677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1716e-f6c7-4f75-ac20-5ff694b62662 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:50.226677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1716e-f6c7-4f75-ac20-5ff694b62662 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:50.233689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:56:50 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:56:50.239939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:50.240385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:50.240522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:50.310721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: INFO dbcounter [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] Registered counter for database keystone Oct 05 13:56:50.332759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG oslo_db.sqlalchemy.engines [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81570) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 13:56:50.344345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writer thread running {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 13:56:50.526405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81570) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 05 13:56:50.526786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG passlib.handlers.bcrypt [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81570) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 05 13:56:50.585181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:56:50.586810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:56:50.719239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:56:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 05 13:56:51.270253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1716e-f6c7-4f75-ac20-5ff694b62662 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:51.347671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: INFO dbcounter [None req-64f1716e-f6c7-4f75-ac20-5ff694b62662 None None] Registered counter for database keystone Oct 05 13:56:51.372990 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG oslo_db.sqlalchemy.engines [None req-64f1716e-f6c7-4f75-ac20-5ff694b62662 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81571) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 05 13:56:51.383904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writer thread running {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 05 13:56:51.502953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1716e-f6c7-4f75-ac20-5ff694b62662 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:51.509900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d179e2c5-72a6-4bad-b065-3330470b1dec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:51.511541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179e2c5-72a6-4bad-b065-3330470b1dec admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:51.511541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179e2c5-72a6-4bad-b065-3330470b1dec admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:51.511794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d179e2c5-72a6-4bad-b065-3330470b1dec admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:51.524773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179e2c5-72a6-4bad-b065-3330470b1dec admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:51.579809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d179e2c5-72a6-4bad-b065-3330470b1dec admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 05 13:56:51.580156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d179e2c5-72a6-4bad-b065-3330470b1dec admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:51.590813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Wed Oct 5 13:56:51 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 13:56:51.597279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:51.618979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e60f0-7779-472d-8c46-c6adbecdaab2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:51.624707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c44f467-dad7-4e01-b154-4c80d55e6101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:51.625871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44f467-dad7-4e01-b154-4c80d55e6101 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:51.626142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44f467-dad7-4e01-b154-4c80d55e6101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:51.626407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c44f467-dad7-4e01-b154-4c80d55e6101 admin admin] PATH_INFO: `/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:51.628127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44f467-dad7-4e01-b154-4c80d55e6101 admin admin] RBAC: Authorizing `identity:get_project(project_id=8abb8ea74a4d492daea0a4d1c426e1f2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:51.679843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING keystone.common.rbac_enforcer.enforcer [None req-1c44f467-dad7-4e01-b154-4c80d55e6101 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 05 13:56:51.680163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c44f467-dad7-4e01-b154-4c80d55e6101 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:51.684758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Wed Oct 5 13:56:51 2022] GET /identity/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:56:53.784529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cec1a-2cc8-44f6-98e0-f177f024c1e6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:53.785444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cec1a-2cc8-44f6-98e0-f177f024c1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:53.785444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cec1a-2cc8-44f6-98e0-f177f024c1e6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:53.792099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:56:53 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:56:53.798842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba184f-2801-485e-8256-af431b03b829 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:53.799283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba184f-2801-485e-8256-af431b03b829 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:53.799406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfba184f-2801-485e-8256-af431b03b829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:53.821340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cfba184f-2801-485e-8256-af431b03b829 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:56:53.822905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cfba184f-2801-485e-8256-af431b03b829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:56:53.831870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:56:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 05 13:56:54.099441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cec1a-2cc8-44f6-98e0-f177f024c1e6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:54.122835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cec1a-2cc8-44f6-98e0-f177f024c1e6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:54.131632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbb8e2-ff28-4f29-a328-e8e4c45594e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:54.132245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbb8e2-ff28-4f29-a328-e8e4c45594e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:54.132524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbb8e2-ff28-4f29-a328-e8e4c45594e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:54.132796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbb8e2-ff28-4f29-a328-e8e4c45594e7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:54.140796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbb8e2-ff28-4f29-a328-e8e4c45594e7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:54.142624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbb8e2-ff28-4f29-a328-e8e4c45594e7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:54.151524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Wed Oct 5 13:56:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 13:56:54.157354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba184f-2801-485e-8256-af431b03b829 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:54.166797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfba184f-2801-485e-8256-af431b03b829 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:54.172254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7f11a9-89c1-4f74-b9f7-3a3f35e18423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:54.172892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f11a9-89c1-4f74-b9f7-3a3f35e18423 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:54.173164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f11a9-89c1-4f74-b9f7-3a3f35e18423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:54.173427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7f11a9-89c1-4f74-b9f7-3a3f35e18423 admin admin] PATH_INFO: `/v3/users/e92af64dbb734bd49c69c6f84550ccd3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:54.174868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f11a9-89c1-4f74-b9f7-3a3f35e18423 admin admin] RBAC: Authorizing `identity:get_user(user_id=e92af64dbb734bd49c69c6f84550ccd3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:54.176708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7f11a9-89c1-4f74-b9f7-3a3f35e18423 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:54.178310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Wed Oct 5 13:56:54 2022] GET /identity/v3/users/e92af64dbb734bd49c69c6f84550ccd3 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:56:56.367164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913e12a-2503-4b21-baa5-a16d1b075630 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:56.367690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913e12a-2503-4b21-baa5-a16d1b075630 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:56.367690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7913e12a-2503-4b21-baa5-a16d1b075630 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:56.369085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:56:56.374409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0800-137e-45d9-b40d-0ebecddbbb5f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:56.374846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0800-137e-45d9-b40d-0ebecddbbb5f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:56.374936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:56.375026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e0800-137e-45d9-b40d-0ebecddbbb5f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:56.375125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:56.375322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:56.376313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:56:56.381382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345fb9f-2876-4ed5-86c7-1ad02b21f059 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:56.381662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345fb9f-2876-4ed5-86c7-1ad02b21f059 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:56.381911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4345fb9f-2876-4ed5-86c7-1ad02b21f059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:56.402680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4345fb9f-2876-4ed5-86c7-1ad02b21f059 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:56:56.403883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4345fb9f-2876-4ed5-86c7-1ad02b21f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:56:56.419811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG passlib.handlers.bcrypt [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81571) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 05 13:56:56.420209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG passlib.handlers.bcrypt [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81571) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 05 13:56:56.468635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:56:56.470258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:56:56.484396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 7/12] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:56:56.572006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 6/13] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:56:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:56:56.745668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345fb9f-2876-4ed5-86c7-1ad02b21f059 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:56.771405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:56:56.846157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:56.853047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4345fb9f-2876-4ed5-86c7-1ad02b21f059 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:56.886468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1896af-956a-40ee-b897-e987ebbcb6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:56.887080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1896af-956a-40ee-b897-e987ebbcb6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:56.887353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1896af-956a-40ee-b897-e987ebbcb6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:56.887630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1896af-956a-40ee-b897-e987ebbcb6b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:56.949986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c04b1d-37fb-4da9-b2e8-a3fc10bd2f3a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:56.952222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:56:56.953815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1896af-956a-40ee-b897-e987ebbcb6b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:56.955299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1896af-956a-40ee-b897-e987ebbcb6b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:56.979629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:56:56.989267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff85ab4-cb85-480a-88fd-c948c7761095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:56.990494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff85ab4-cb85-480a-88fd-c948c7761095 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:56.990871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff85ab4-cb85-480a-88fd-c948c7761095 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:56.991037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff85ab4-cb85-480a-88fd-c948c7761095 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:57.003109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff85ab4-cb85-480a-88fd-c948c7761095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2ff85ab4-cb85-480a-88fd-c948c7761095 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:56:57.011529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:56:57.012407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:56:57.013195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:56:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:56:57.018937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff85ab4-cb85-480a-88fd-c948c7761095 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:57.056653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 8/15] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:56:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:56:57.062517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1896af-956a-40ee-b897-e987ebbcb6b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:57.117490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff85ab4-cb85-480a-88fd-c948c7761095 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:57.149718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:57.150322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:57.150596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:57.150862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:57.151525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:56:57.151823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:57.153262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:57.160973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1896af-956a-40ee-b897-e987ebbcb6b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:57.164670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 8/16] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:56:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:56:57.170579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:57.185115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:56:57.196176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:56:57.197749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:57.198563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:57.198845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:57.199116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:57.209007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:57.210507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:56:57.229749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:56:57.230762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:56:57.231567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:56:57.231567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:56:57.231567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:56:57.231567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:56:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:56:57.237057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:57.268344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2c5806-eab1-4c6b-9f86-9fea806d6fd0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:57.302008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9092cd-9dc5-453d-9c3a-185e5cc44da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:57.302691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9092cd-9dc5-453d-9c3a-185e5cc44da2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:57.302968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9092cd-9dc5-453d-9c3a-185e5cc44da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:57.303236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9092cd-9dc5-453d-9c3a-185e5cc44da2 None admin] PATH_INFO: `/v3/users/e92af64dbb734bd49c69c6f84550ccd3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:57.304864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9092cd-9dc5-453d-9c3a-185e5cc44da2 None admin] RBAC: Authorizing `identity:get_user(user_id=e92af64dbb734bd49c69c6f84550ccd3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:57.306358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9092cd-9dc5-453d-9c3a-185e5cc44da2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:57.307941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:56:57 2022] GET /identity/v3/users/e92af64dbb734bd49c69c6f84550ccd3 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:56:57.313450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9092cd-9dc5-453d-9c3a-185e5cc44da2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:57.334674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ac310c-1775-48eb-b50a-116fb889713b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:57.368016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:57.368635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:57.368914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:57.369183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:57.369874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:56:57.370172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:57.371577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:57.388519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:56:57.408651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:56:57.417007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9092cd-9dc5-453d-9c3a-185e5cc44da2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:57.454370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d53492-4a04-4188-8945-ee20c1b7a639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:57.455039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53492-4a04-4188-8945-ee20c1b7a639 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:57.455316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53492-4a04-4188-8945-ee20c1b7a639 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:57.455565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d53492-4a04-4188-8945-ee20c1b7a639 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:57.456802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d53492-4a04-4188-8945-ee20c1b7a639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:57.458375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d53492-4a04-4188-8945-ee20c1b7a639 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:57.460027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:56:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:56:57.466490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:56:57.507873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:56:57.582378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660636a9-126f-473c-a766-bdadd2a1d55e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:56:57.616889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae803a96-7643-4f88-babf-e08ab18ffd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:56:57.617939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803a96-7643-4f88-babf-e08ab18ffd89 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:57.618312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803a96-7643-4f88-babf-e08ab18ffd89 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:57.618763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae803a96-7643-4f88-babf-e08ab18ffd89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:57.619547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae803a96-7643-4f88-babf-e08ab18ffd89 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.domain.id=default, user.id=e92af64dbb734bd49c69c6f84550ccd3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:56:57.619861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae803a96-7643-4f88-babf-e08ab18ffd89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:56:57.621395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae803a96-7643-4f88-babf-e08ab18ffd89 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:56:57.630554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 11/21] 10.222.0.23 () {56 vars in 1322 bytes} [Wed Oct 5 13:56:57 2022] GET /identity/v3/role_assignments?user.id=e92af64dbb734bd49c69c6f84550ccd3&scope.domain.id=default&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 233 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:56:59.855653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33bd97-8fec-4dcb-b196-dc8d2f9d84c1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:59.856161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33bd97-8fec-4dcb-b196-dc8d2f9d84c1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:59.856161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33bd97-8fec-4dcb-b196-dc8d2f9d84c1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:59.857767 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:56:59.863112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d066c08-9664-4082-9f0d-d0136101e701 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:56:59.863531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d066c08-9664-4082-9f0d-d0136101e701 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:56:59.863694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d066c08-9664-4082-9f0d-d0136101e701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:56:59.890647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0d066c08-9664-4082-9f0d-d0136101e701 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:56:59.891837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0d066c08-9664-4082-9f0d-d0136101e701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:56:59.979975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 12/23] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:56:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:00.095633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa569-8780-40eb-9114-7a1a5ec093c0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:00.095973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa569-8780-40eb-9114-7a1a5ec093c0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:00.096172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35fa569-8780-40eb-9114-7a1a5ec093c0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:00.097521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:00.103422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2752d7d-aee2-4db9-9831-8a2bb83dce7e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:00.103705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2752d7d-aee2-4db9-9831-8a2bb83dce7e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:00.103972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2752d7d-aee2-4db9-9831-8a2bb83dce7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:00.130862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c2752d7d-aee2-4db9-9831-8a2bb83dce7e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:00.131992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c2752d7d-aee2-4db9-9831-8a2bb83dce7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:00.160640 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:00.223741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:00.469755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa569-8780-40eb-9114-7a1a5ec093c0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:00.587267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35fa569-8780-40eb-9114-7a1a5ec093c0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:00.630045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:00.630736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:00.631012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:00.631282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:00.639131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729229fd-63fa-4b68-9d6d-4c6af6600180 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:00.639409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729229fd-63fa-4b68-9d6d-4c6af6600180 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:00.639681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729229fd-63fa-4b68-9d6d-4c6af6600180 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:00.641046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 14/26] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:00.646504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ce4d-84d5-4d91-afa3-d9bb9e005a49 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:00.646787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ce4d-84d5-4d91-afa3-d9bb9e005a49 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:00.647063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f5ce4d-84d5-4d91-afa3-d9bb9e005a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:00.667853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:00.669341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:00.687531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-66f5ce4d-84d5-4d91-afa3-d9bb9e005a49 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:00.689138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-66f5ce4d-84d5-4d91-afa3-d9bb9e005a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:57:00.697587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:00.698543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:57:00.699346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:57:00.699346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:57:00.699346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:00.699346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 13/27] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:00.707145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:00.732545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:00.805983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 15/28] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:00.807096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5ce4d-84d5-4d91-afa3-d9bb9e005a49 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:00.901840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f908ee-e9b2-4c70-bd62-ccd7b9f198a4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:00.943429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424988b-8d97-42f6-991c-00ccee286b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:00.944359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424988b-8d97-42f6-991c-00ccee286b42 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:00.944766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424988b-8d97-42f6-991c-00ccee286b42 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:00.945180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424988b-8d97-42f6-991c-00ccee286b42 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:00.946122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424988b-8d97-42f6-991c-00ccee286b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:00.946556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424988b-8d97-42f6-991c-00ccee286b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:00.948154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424988b-8d97-42f6-991c-00ccee286b42 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:00.962011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:00.969155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 14/29] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:00.975100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1681c-6d72-403a-8db8-1cce06f915db None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:00.975545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1681c-6d72-403a-8db8-1cce06f915db None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:00.975969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1681c-6d72-403a-8db8-1cce06f915db None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:00.977706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 15/30] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:00.979612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1681c-6d72-403a-8db8-1cce06f915db None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:00.995033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f5ce4d-84d5-4d91-afa3-d9bb9e005a49 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:01.065143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:01.081351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d6d957-f2e6-4637-9dad-40a1a8809a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:01.082037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6d957-f2e6-4637-9dad-40a1a8809a8d None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.082312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6d957-f2e6-4637-9dad-40a1a8809a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.082579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d6d957-f2e6-4637-9dad-40a1a8809a8d None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.083291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6d957-f2e6-4637-9dad-40a1a8809a8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:01.084701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d6d957-f2e6-4637-9dad-40a1a8809a8d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:01.101170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1681c-6d72-403a-8db8-1cce06f915db None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:01.139069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f6bb-045b-47a9-aa16-410819df8ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:01.140131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f6bb-045b-47a9-aa16-410819df8ade None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.140593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f6bb-045b-47a9-aa16-410819df8ade None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.141041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f6bb-045b-47a9-aa16-410819df8ade None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002f6bb-045b-47a9-aa16-410819df8ade None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:01.143779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002f6bb-045b-47a9-aa16-410819df8ade None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:01.222926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 16/31] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 13:57:00 2022] POST /identity/v3/roles => generated 219 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:01.224577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a665ec1-c26f-46a2-b641-2824a2fd8f67 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.224867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a665ec1-c26f-46a2-b641-2824a2fd8f67 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.225144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a665ec1-c26f-46a2-b641-2824a2fd8f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.245457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2a665ec1-c26f-46a2-b641-2824a2fd8f67 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:01.246921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a665ec1-c26f-46a2-b641-2824a2fd8f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:01.298591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 16/32] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:57:00 2022] POST /identity/v3/projects => generated 301 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:01.300122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f6bb-045b-47a9-aa16-410819df8ade None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:01.322301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 17/33] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:01.424505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f6bb-045b-47a9-aa16-410819df8ade None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:01.454873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:01.464418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3d242-16ab-4c2e-8b87-8f1119b68d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:01.465115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3d242-16ab-4c2e-8b87-8f1119b68d08 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.465818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3d242-16ab-4c2e-8b87-8f1119b68d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.466438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3d242-16ab-4c2e-8b87-8f1119b68d08 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.467157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3d242-16ab-4c2e-8b87-8f1119b68d08 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:01.469217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3d242-16ab-4c2e-8b87-8f1119b68d08 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:01.499290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 17/34] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 13:57:01 2022] POST /identity/v3/roles => generated 225 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:01.541568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060ef98-9a79-49e0-a128-e9c3b0052a35 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.542283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060ef98-9a79-49e0-a128-e9c3b0052a35 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.542780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060ef98-9a79-49e0-a128-e9c3b0052a35 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.545793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 18/35] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:01.550692 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd9311-9011-4c67-a88b-f4e2c2f02559 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.551025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd9311-9011-4c67-a88b-f4e2c2f02559 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.551252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cd9311-9011-4c67-a88b-f4e2c2f02559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.583068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-62cd9311-9011-4c67-a88b-f4e2c2f02559 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:01.584780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-62cd9311-9011-4c67-a88b-f4e2c2f02559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:01.684019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 18/36] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:01.689757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060ef98-9a79-49e0-a128-e9c3b0052a35 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:01.782339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:01.783194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da814125-c4f3-4597-ba64-8689c674daa6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.783482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da814125-c4f3-4597-ba64-8689c674daa6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.783760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da814125-c4f3-4597-ba64-8689c674daa6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.785424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 19/37] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:01.791056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cc3ad-8e4c-4279-a727-ad60316e5cc0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.791338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cc3ad-8e4c-4279-a727-ad60316e5cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.791613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322cc3ad-8e4c-4279-a727-ad60316e5cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.817146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060ef98-9a79-49e0-a128-e9c3b0052a35 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:01.825734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-322cc3ad-8e4c-4279-a727-ad60316e5cc0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:01.826881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-322cc3ad-8e4c-4279-a727-ad60316e5cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:01.871053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b268c-1dd1-4d29-b002-a79e0c6f411d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:01.872055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b268c-1dd1-4d29-b002-a79e0c6f411d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:01.872530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b268c-1dd1-4d29-b002-a79e0c6f411d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:01.872943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b268c-1dd1-4d29-b002-a79e0c6f411d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:01.887337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b268c-1dd1-4d29-b002-a79e0c6f411d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-af7b268c-1dd1-4d29-b002-a79e0c6f411d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:01.895037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:01.895912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:01.896702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 19/38] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:01.903520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b268c-1dd1-4d29-b002-a79e0c6f411d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:01.922667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 20/39] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:01.982966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cc3ad-8e4c-4279-a727-ad60316e5cc0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:02.036197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:02.099637 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322cc3ad-8e4c-4279-a727-ad60316e5cc0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:02.118684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b268c-1dd1-4d29-b002-a79e0c6f411d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:02.138311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7d5b5-0f14-420a-aeea-3c81364c1270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:02.138931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7d5b5-0f14-420a-aeea-3c81364c1270 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:02.139628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7d5b5-0f14-420a-aeea-3c81364c1270 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:02.139908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7d5b5-0f14-420a-aeea-3c81364c1270 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:02.140591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7d5b5-0f14-420a-aeea-3c81364c1270 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:02.142017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe7d5b5-0f14-420a-aeea-3c81364c1270 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:02.154744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:02.155674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:02.156148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:02.156611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:02.157493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:02.157603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 21/40] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 13:57:01 2022] POST /identity/v3/roles => generated 223 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:57:02.158142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:02.159718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:02.169797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:02.180138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 20/41] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:02.181193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7d5b5-0f14-420a-aeea-3c81364c1270 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:02.185977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:02.296790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8d1d00-dbab-45eb-99ac-b93574998f94 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:02.300924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe7d5b5-0f14-420a-aeea-3c81364c1270 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:02.309560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:02.339028 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ed453-a39a-4d18-aa31-e86f96b8e68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:02.339691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed453-a39a-4d18-aa31-e86f96b8e68f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:02.339962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed453-a39a-4d18-aa31-e86f96b8e68f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:02.340230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9ed453-a39a-4d18-aa31-e86f96b8e68f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:02.341430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ed453-a39a-4d18-aa31-e86f96b8e68f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:02.343064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9ed453-a39a-4d18-aa31-e86f96b8e68f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:02.343724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dab0e6-253f-4e05-b6c0-53e449c7e75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:02.345222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dab0e6-253f-4e05-b6c0-53e449c7e75b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:02.345329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 21/42] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:02.345399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dab0e6-253f-4e05-b6c0-53e449c7e75b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:02.345399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dab0e6-253f-4e05-b6c0-53e449c7e75b None admin] PATH_INFO: `/v3/users/e92af64dbb734bd49c69c6f84550ccd3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:02.346869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dab0e6-253f-4e05-b6c0-53e449c7e75b None admin] RBAC: Authorizing `identity:get_user(user_id=e92af64dbb734bd49c69c6f84550ccd3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:02.348353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dab0e6-253f-4e05-b6c0-53e449c7e75b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:02.350087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 22/43] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:02 2022] GET /identity/v3/users/e92af64dbb734bd49c69c6f84550ccd3 => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:02.350580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ed453-a39a-4d18-aa31-e86f96b8e68f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:02.356512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dab0e6-253f-4e05-b6c0-53e449c7e75b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:02.428630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:02.462172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9ed453-a39a-4d18-aa31-e86f96b8e68f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:02.462527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dab0e6-253f-4e05-b6c0-53e449c7e75b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:02.495586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12288c70-f4eb-41bc-b5d0-467328bdab6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:02.496232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12288c70-f4eb-41bc-b5d0-467328bdab6e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:02.496512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12288c70-f4eb-41bc-b5d0-467328bdab6e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:02.496736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a273d-0911-4a72-9399-73cc0ac77a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:02.496834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12288c70-f4eb-41bc-b5d0-467328bdab6e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:02.497347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a273d-0911-4a72-9399-73cc0ac77a0b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:02.497625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a273d-0911-4a72-9399-73cc0ac77a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:02.497924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a273d-0911-4a72-9399-73cc0ac77a0b None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:02.498024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12288c70-f4eb-41bc-b5d0-467328bdab6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:02.498574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a273d-0911-4a72-9399-73cc0ac77a0b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:02.499376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12288c70-f4eb-41bc-b5d0-467328bdab6e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:02.500012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a273d-0911-4a72-9399-73cc0ac77a0b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:02.501119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 23/44] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:02.507564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12288c70-f4eb-41bc-b5d0-467328bdab6e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:02.531263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 22/45] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:57:02 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:57:02.552168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:02.615516 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12288c70-f4eb-41bc-b5d0-467328bdab6e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:02.652291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcbe05d-ebc4-4591-9d04-6dcdae2b4965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:02.652991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe05d-ebc4-4591-9d04-6dcdae2b4965 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:02.653242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe05d-ebc4-4591-9d04-6dcdae2b4965 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:02.653522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcbe05d-ebc4-4591-9d04-6dcdae2b4965 None admin] PATH_INFO: `/v3/domains/default/users/e92af64dbb734bd49c69c6f84550ccd3/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:02.655667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbe05d-ebc4-4591-9d04-6dcdae2b4965 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e92af64dbb734bd49c69c6f84550ccd3, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:02.657144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcbe05d-ebc4-4591-9d04-6dcdae2b4965 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:02.675095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 24/46] 10.222.0.23 () {58 vars in 1361 bytes} [Wed Oct 5 13:57:02 2022] PUT /identity/v3/domains/default/users/e92af64dbb734bd49c69c6f84550ccd3/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:04.691633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d20a0-7ca0-4ae7-864f-5edb367ece00 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:04.692208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d20a0-7ca0-4ae7-864f-5edb367ece00 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:04.692208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d20a0-7ca0-4ae7-864f-5edb367ece00 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:04.693612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 23/47] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:04.699041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4768099-c18d-4f46-9c23-10ec2261835d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:04.699447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4768099-c18d-4f46-9c23-10ec2261835d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:04.699607 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4768099-c18d-4f46-9c23-10ec2261835d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:04.718372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a4768099-c18d-4f46-9c23-10ec2261835d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:04.719579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a4768099-c18d-4f46-9c23-10ec2261835d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:04.759251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f160e1-6640-4ba6-86b6-a043575f2e0b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:04.759427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f160e1-6640-4ba6-86b6-a043575f2e0b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:04.759713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f160e1-6640-4ba6-86b6-a043575f2e0b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:04.761106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 24/48] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:04.766219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da1f3-464a-470b-8958-636373f58a72 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:04.766502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da1f3-464a-470b-8958-636373f58a72 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:04.766779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33da1f3-464a-470b-8958-636373f58a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:04.791082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f33da1f3-464a-470b-8958-636373f58a72 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:04.792330 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f33da1f3-464a-470b-8958-636373f58a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:04.793393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 25/49] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:04.861030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 25/50] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:05.059612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4768099-c18d-4f46-9c23-10ec2261835d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:05.090439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:05.125948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33da1f3-464a-470b-8958-636373f58a72 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:05.181418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:05.186303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4768099-c18d-4f46-9c23-10ec2261835d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:05.222270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f42f-364e-4878-9aa7-273a7e4bcd57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:05.222912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f42f-364e-4878-9aa7-273a7e4bcd57 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:05.223194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f42f-364e-4878-9aa7-273a7e4bcd57 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:05.223453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3381f42f-364e-4878-9aa7-273a7e4bcd57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:05.224627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381f42f-364e-4878-9aa7-273a7e4bcd57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:05.226136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3381f42f-364e-4878-9aa7-273a7e4bcd57 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:05.227718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 26/51] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:05.232097 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33da1f3-464a-470b-8958-636373f58a72 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:05.233412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f42f-364e-4878-9aa7-273a7e4bcd57 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:05.265937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64696b34-a8e9-4c65-858b-b8a0db8bfec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:05.266584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64696b34-a8e9-4c65-858b-b8a0db8bfec0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:05.266853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64696b34-a8e9-4c65-858b-b8a0db8bfec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:05.267124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64696b34-a8e9-4c65-858b-b8a0db8bfec0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:05.277174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64696b34-a8e9-4c65-858b-b8a0db8bfec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-64696b34-a8e9-4c65-858b-b8a0db8bfec0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:05.281962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:05.282798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:05.283540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 26/52] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:05.288721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64696b34-a8e9-4c65-858b-b8a0db8bfec0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:05.325621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:05.328941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3381f42f-364e-4878-9aa7-273a7e4bcd57 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:05.365097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d85a0b-b95b-4d38-8818-3b76103f73f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:05.365785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d85a0b-b95b-4d38-8818-3b76103f73f2 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:05.366058 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d85a0b-b95b-4d38-8818-3b76103f73f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:05.366329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d85a0b-b95b-4d38-8818-3b76103f73f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:05.366975 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d85a0b-b95b-4d38-8818-3b76103f73f2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:05.368444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d85a0b-b95b-4d38-8818-3b76103f73f2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:05.387042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64696b34-a8e9-4c65-858b-b8a0db8bfec0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:05.397756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 27/53] 10.222.0.23 () {60 vars in 1140 bytes} [Wed Oct 5 13:57:05 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:57:05.398465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:05.425476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:05.426186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:05.426469 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:05.426742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:05.427472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:05.427779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:05.429252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:05.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 27/54] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:05.445269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d85a0b-b95b-4d38-8818-3b76103f73f2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:05.676426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d85a0b-b95b-4d38-8818-3b76103f73f2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:05.712711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f7824-0e8d-479a-a76b-ea4a3fb3fd33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:05.713358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7824-0e8d-479a-a76b-ea4a3fb3fd33 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:05.713592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7824-0e8d-479a-a76b-ea4a3fb3fd33 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:05.713914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242f7824-0e8d-479a-a76b-ea4a3fb3fd33 None admin] PATH_INFO: `/v3/users/e92af64dbb734bd49c69c6f84550ccd3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:05.715098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f7824-0e8d-479a-a76b-ea4a3fb3fd33 None admin] RBAC: Authorizing `identity:get_user(user_id=e92af64dbb734bd49c69c6f84550ccd3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:05.716539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242f7824-0e8d-479a-a76b-ea4a3fb3fd33 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:05.718151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 28/55] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:05 2022] GET /identity/v3/users/e92af64dbb734bd49c69c6f84550ccd3 => generated 247 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:05.723942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:05.829800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c60e75-bc38-4a3a-86f9-20817bb3d8b0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:05.864134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f787d98b-a278-4f8b-8fe0-386734672d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:05.864839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d98b-a278-4f8b-8fe0-386734672d57 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:05.865112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d98b-a278-4f8b-8fe0-386734672d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:05.865382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f787d98b-a278-4f8b-8fe0-386734672d57 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:05.866546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787d98b-a278-4f8b-8fe0-386734672d57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:05.868038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f787d98b-a278-4f8b-8fe0-386734672d57 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:05.869618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 28/56] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:05.875660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f7824-0e8d-479a-a76b-ea4a3fb3fd33 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:05.900374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:05.982919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242f7824-0e8d-479a-a76b-ea4a3fb3fd33 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:06.017330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdf4f26-4952-4a5f-8a80-ca972fb240f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:06.017980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf4f26-4952-4a5f-8a80-ca972fb240f3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:06.018268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf4f26-4952-4a5f-8a80-ca972fb240f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:06.018539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdf4f26-4952-4a5f-8a80-ca972fb240f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:06.019267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf4f26-4952-4a5f-8a80-ca972fb240f3 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.domain.id=default, user.id=e92af64dbb734bd49c69c6f84550ccd3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:06.019576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf4f26-4952-4a5f-8a80-ca972fb240f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:06.021272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdf4f26-4952-4a5f-8a80-ca972fb240f3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:06.030336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 29/57] 10.222.0.23 () {56 vars in 1322 bytes} [Wed Oct 5 13:57:05 2022] GET /identity/v3/role_assignments?user.id=e92af64dbb734bd49c69c6f84550ccd3&scope.domain.id=default&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 530 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:07.577413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11bad1-5a98-4066-819f-006a2cce243d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:07.578283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11bad1-5a98-4066-819f-006a2cce243d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:07.578417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11bad1-5a98-4066-819f-006a2cce243d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:07.580620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:07.586134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eedfa9-f146-4f92-8393-6b86a157b800 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:07.586545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eedfa9-f146-4f92-8393-6b86a157b800 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:07.586654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47eedfa9-f146-4f92-8393-6b86a157b800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:07.606264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-47eedfa9-f146-4f92-8393-6b86a157b800 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:07.607458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47eedfa9-f146-4f92-8393-6b86a157b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:07.676042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:07.944263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11bad1-5a98-4066-819f-006a2cce243d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:07.996118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:08.059022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11bad1-5a98-4066-819f-006a2cce243d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:08.094955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74f01c-4f50-4e80-b7ab-0125fe2d02f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:08.095590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74f01c-4f50-4e80-b7ab-0125fe2d02f3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:08.095876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74f01c-4f50-4e80-b7ab-0125fe2d02f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:08.096150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74f01c-4f50-4e80-b7ab-0125fe2d02f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:08.097317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74f01c-4f50-4e80-b7ab-0125fe2d02f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:08.098865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c74f01c-4f50-4e80-b7ab-0125fe2d02f3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:08.100412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 30/60] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:08.109230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eedfa9-f146-4f92-8393-6b86a157b800 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:08.127039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:08.213475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47eedfa9-f146-4f92-8393-6b86a157b800 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:08.257758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcf31fa-1ba0-4127-b810-d4f6d994e945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:08.258771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf31fa-1ba0-4127-b810-d4f6d994e945 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:08.259199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf31fa-1ba0-4127-b810-d4f6d994e945 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:08.259597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcf31fa-1ba0-4127-b810-d4f6d994e945 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:08.260605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf31fa-1ba0-4127-b810-d4f6d994e945 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:08.263106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcf31fa-1ba0-4127-b810-d4f6d994e945 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:08.303048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 31/61] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:57:08 2022] POST /identity/v3/users => generated 275 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:11.644685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf4dc9-64c6-478d-ad0b-2f0e02a18fbf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:11.645789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf4dc9-64c6-478d-ad0b-2f0e02a18fbf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:11.646333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf4dc9-64c6-478d-ad0b-2f0e02a18fbf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:11.648138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 31/62] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:11.653800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68a5c-ceb5-4908-a7b3-f424b6f582c7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:11.654224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68a5c-ceb5-4908-a7b3-f424b6f582c7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:11.654474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd68a5c-ceb5-4908-a7b3-f424b6f582c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:11.674891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19ded6-3c27-4dfd-82f4-5cc92db9de8b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:11.675215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19ded6-3c27-4dfd-82f4-5cc92db9de8b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:11.675431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad19ded6-3c27-4dfd-82f4-5cc92db9de8b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:11.676767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 32/63] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:11.681999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530bcd4-e7e4-4c4c-9211-62376e3f74e3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:11.682278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530bcd4-e7e4-4c4c-9211-62376e3f74e3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:11.682549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a530bcd4-e7e4-4c4c-9211-62376e3f74e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:11.690834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1bd68a5c-ceb5-4908-a7b3-f424b6f582c7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:11.692024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1bd68a5c-ceb5-4908-a7b3-f424b6f582c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:11.713044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a530bcd4-e7e4-4c4c-9211-62376e3f74e3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:11.714282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a530bcd4-e7e4-4c4c-9211-62376e3f74e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:11.826190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:11.832389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 32/64] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:11.838406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 33/65] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:11.897826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655728b-6de9-4567-b5cb-9221731abac1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:11.898205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655728b-6de9-4567-b5cb-9221731abac1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:11.898390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655728b-6de9-4567-b5cb-9221731abac1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:11.899931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 33/66] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:11.905475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf72aaf-5903-4aee-b29c-f445d32916e9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:11.905789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf72aaf-5903-4aee-b29c-f445d32916e9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:11.906071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf72aaf-5903-4aee-b29c-f445d32916e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:11.934687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-faf72aaf-5903-4aee-b29c-f445d32916e9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:11.935893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-faf72aaf-5903-4aee-b29c-f445d32916e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:11.973619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:12.027921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 34/67] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:12.136421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655728b-6de9-4567-b5cb-9221731abac1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.248915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ee289-0304-4faa-aed5-01b33084893f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.249769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ee289-0304-4faa-aed5-01b33084893f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.250248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187ee289-0304-4faa-aed5-01b33084893f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.254009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 35/68] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:12.258058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544076e3-b916-4ed8-bad9-52960bddcce8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.258674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544076e3-b916-4ed8-bad9-52960bddcce8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.259160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544076e3-b916-4ed8-bad9-52960bddcce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.268952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655728b-6de9-4567-b5cb-9221731abac1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:12.282150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-544076e3-b916-4ed8-bad9-52960bddcce8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:12.283338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-544076e3-b916-4ed8-bad9-52960bddcce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:12.304927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50f49-554c-475c-a681-1b743e642ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:12.305923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50f49-554c-475c-a681-1b743e642ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.306459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50f49-554c-475c-a681-1b743e642ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.307036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb50f49-554c-475c-a681-1b743e642ed8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.317950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb50f49-554c-475c-a681-1b743e642ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dbb50f49-554c-475c-a681-1b743e642ed8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:12.323029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:12.323923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:12.324656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 34/69] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:12.325256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50f49-554c-475c-a681-1b743e642ed8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.356297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 36/70] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:12.357536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544076e3-b916-4ed8-bad9-52960bddcce8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.393955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:12.446308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb50f49-554c-475c-a681-1b743e642ed8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:12.461817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:12.466381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544076e3-b916-4ed8-bad9-52960bddcce8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:12.490026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb5595-0eff-4261-a9d3-242561b43caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:12.491095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb5595-0eff-4261-a9d3-242561b43caa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.491375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb5595-0eff-4261-a9d3-242561b43caa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.491645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb5595-0eff-4261-a9d3-242561b43caa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.506198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb5595-0eff-4261-a9d3-242561b43caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a6fb5595-0eff-4261-a9d3-242561b43caa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:12.509385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:12.510378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:12.511120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:12.511678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc9740-824d-4661-9d44-d1ab36202334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:12.511678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc9740-824d-4661-9d44-d1ab36202334 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.511825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 35/71] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:12.511889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc9740-824d-4661-9d44-d1ab36202334 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.511889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc9740-824d-4661-9d44-d1ab36202334 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.512461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc9740-824d-4661-9d44-d1ab36202334 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:12.512461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc9740-824d-4661-9d44-d1ab36202334 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:12.513865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc9740-824d-4661-9d44-d1ab36202334 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:12.515941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb5595-0eff-4261-a9d3-242561b43caa None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.528238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 37/72] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:12.528238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a171ac-fd0c-447b-877b-fcafa1e51463 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.528238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a171ac-fd0c-447b-877b-fcafa1e51463 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.528238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a171ac-fd0c-447b-877b-fcafa1e51463 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.528238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 38/73] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:12.529272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a171ac-fd0c-447b-877b-fcafa1e51463 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.625985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb5595-0eff-4261-a9d3-242561b43caa None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:12.638062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a171ac-fd0c-447b-877b-fcafa1e51463 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:12.645779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:12.660234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d48b-283c-4490-a6d0-1875b2321d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:12.660871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d48b-283c-4490-a6d0-1875b2321d51 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.661168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d48b-283c-4490-a6d0-1875b2321d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.661426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097d48b-283c-4490-a6d0-1875b2321d51 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.671453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097d48b-283c-4490-a6d0-1875b2321d51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:12.672305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417ff790-7688-4a44-a649-76bf233cd0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:12.672921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ff790-7688-4a44-a649-76bf233cd0ea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.673196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ff790-7688-4a44-a649-76bf233cd0ea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.673464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417ff790-7688-4a44-a649-76bf233cd0ea None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.674161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ff790-7688-4a44-a649-76bf233cd0ea None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:12.674463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ff790-7688-4a44-a649-76bf233cd0ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:12.675850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417ff790-7688-4a44-a649-76bf233cd0ea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b097d48b-283c-4490-a6d0-1875b2321d51 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:12.676161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:12.676935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 36/74] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:12.682128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d48b-283c-4490-a6d0-1875b2321d51 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.685153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 39/75] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:12.689794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206973c-215a-4b46-8caf-12baf43404c8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.689794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206973c-215a-4b46-8caf-12baf43404c8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.689794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a206973c-215a-4b46-8caf-12baf43404c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.705840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a206973c-215a-4b46-8caf-12baf43404c8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:12.707056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a206973c-215a-4b46-8caf-12baf43404c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:12.723068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:12.773537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 40/76] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:12.774332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206973c-215a-4b46-8caf-12baf43404c8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.782515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097d48b-283c-4490-a6d0-1875b2321d51 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:12.817542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264554f-5811-438e-8a4a-6113d39a7642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:12.818280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264554f-5811-438e-8a4a-6113d39a7642 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.818555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264554f-5811-438e-8a4a-6113d39a7642 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.818822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8264554f-5811-438e-8a4a-6113d39a7642 None admin] PATH_INFO: `/v3/users/e92af64dbb734bd49c69c6f84550ccd3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.819955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264554f-5811-438e-8a4a-6113d39a7642 None admin] RBAC: Authorizing `identity:get_user(user_id=e92af64dbb734bd49c69c6f84550ccd3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:12.821406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8264554f-5811-438e-8a4a-6113d39a7642 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:12.822996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 37/77] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/users/e92af64dbb734bd49c69c6f84550ccd3 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:12.823753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264554f-5811-438e-8a4a-6113d39a7642 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.874404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:12.882806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a206973c-215a-4b46-8caf-12baf43404c8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:12.919031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f83351-0c6b-401b-9c7f-2ca9050858df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:12.919682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f83351-0c6b-401b-9c7f-2ca9050858df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.919958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f83351-0c6b-401b-9c7f-2ca9050858df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.920234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f83351-0c6b-401b-9c7f-2ca9050858df None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.929362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8264554f-5811-438e-8a4a-6113d39a7642 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:12.930787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f83351-0c6b-401b-9c7f-2ca9050858df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b1f83351-0c6b-401b-9c7f-2ca9050858df None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:12.935601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:12.936606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 41/78] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:12.937555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f83351-0c6b-401b-9c7f-2ca9050858df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:12.956110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:12.972091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:12.972716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:12.973147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:12.973435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:12.974521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:12.974837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:12.976269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:12.984782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 38/79] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:12.985585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.048397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f83351-0c6b-401b-9c7f-2ca9050858df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.082027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755376b-0396-454a-a156-7fffaeeaaea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.082660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755376b-0396-454a-a156-7fffaeeaaea2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.082935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755376b-0396-454a-a156-7fffaeeaaea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.083218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d755376b-0396-454a-a156-7fffaeeaaea2 None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.087052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012a5f00-b4d4-477f-b57d-52bdbd2b4a0e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.096737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755376b-0396-454a-a156-7fffaeeaaea2 None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.097147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:13.098943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d755376b-0396-454a-a156-7fffaeeaaea2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.100695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 42/80] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:13.101464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755376b-0396-454a-a156-7fffaeeaaea2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.120852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a5b48-fc07-45f0-af0c-4ad729059e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.121471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5b48-fc07-45f0-af0c-4ad729059e5d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.121783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5b48-fc07-45f0-af0c-4ad729059e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.122056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a5b48-fc07-45f0-af0c-4ad729059e5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.174376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:13.188412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a5b48-fc07-45f0-af0c-4ad729059e5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.189925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a5b48-fc07-45f0-af0c-4ad729059e5d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.204368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d755376b-0396-454a-a156-7fffaeeaaea2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.237318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.237953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.238238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.238494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.239578 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:13.239965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.241508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.249492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 43/81] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:13.250311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.285063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 39/82] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:13.285962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a5b48-fc07-45f0-af0c-4ad729059e5d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.296994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:13.347255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a7e11-bd93-4833-9cc3-120f1d2e5375 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.375413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:13.381036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4b10d-18d0-4077-a5e9-f4a73fd5f6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.381786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4b10d-18d0-4077-a5e9-f4a73fd5f6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.381924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a5b48-fc07-45f0-af0c-4ad729059e5d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.382191 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4b10d-18d0-4077-a5e9-f4a73fd5f6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.382530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce4b10d-18d0-4077-a5e9-f4a73fd5f6e4 None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.383755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4b10d-18d0-4077-a5e9-f4a73fd5f6e4 None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.385235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce4b10d-18d0-4077-a5e9-f4a73fd5f6e4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.386882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 44/83] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:13 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:13.387655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4b10d-18d0-4077-a5e9-f4a73fd5f6e4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.416043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d5e41-0cc1-49a9-84e1-0d83ee61a2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.416685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d5e41-0cc1-49a9-84e1-0d83ee61a2af None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.416961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d5e41-0cc1-49a9-84e1-0d83ee61a2af None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.417240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d5e41-0cc1-49a9-84e1-0d83ee61a2af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.418399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d5e41-0cc1-49a9-84e1-0d83ee61a2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.419892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d5e41-0cc1-49a9-84e1-0d83ee61a2af None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.421473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 40/84] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:13.422292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d5e41-0cc1-49a9-84e1-0d83ee61a2af None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.495483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce4b10d-18d0-4077-a5e9-f4a73fd5f6e4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.521618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:13.530189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda05cc2-4aa7-4964-b43b-2203ad6a958f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.530932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda05cc2-4aa7-4964-b43b-2203ad6a958f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.531272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda05cc2-4aa7-4964-b43b-2203ad6a958f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.531604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda05cc2-4aa7-4964-b43b-2203ad6a958f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.541272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d5e41-0cc1-49a9-84e1-0d83ee61a2af None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.579297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bf0d9-d70e-4a24-ab07-77a3a065edeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.579950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf0d9-d70e-4a24-ab07-77a3a065edeb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.580231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf0d9-d70e-4a24-ab07-77a3a065edeb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.580514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf0d9-d70e-4a24-ab07-77a3a065edeb None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.581725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bf0d9-d70e-4a24-ab07-77a3a065edeb None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.583231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37bf0d9-d70e-4a24-ab07-77a3a065edeb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.584905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 41/85] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:13 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:13.585885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bf0d9-d70e-4a24-ab07-77a3a065edeb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.600235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:13.603691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda05cc2-4aa7-4964-b43b-2203ad6a958f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.605731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda05cc2-4aa7-4964-b43b-2203ad6a958f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.688069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37bf0d9-d70e-4a24-ab07-77a3a065edeb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.710872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 45/86] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:13.712240 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda05cc2-4aa7-4964-b43b-2203ad6a958f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.726055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92e941-9c2d-4d88-8041-1ca3b049781c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.726888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92e941-9c2d-4d88-8041-1ca3b049781c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.727221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92e941-9c2d-4d88-8041-1ca3b049781c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.727512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92e941-9c2d-4d88-8041-1ca3b049781c None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.728688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92e941-9c2d-4d88-8041-1ca3b049781c None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.730312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92e941-9c2d-4d88-8041-1ca3b049781c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.737157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 42/87] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:13 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:13.737157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92e941-9c2d-4d88-8041-1ca3b049781c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.754114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:13.820842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:13.823555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda05cc2-4aa7-4964-b43b-2203ad6a958f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.838289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92e941-9c2d-4d88-8041-1ca3b049781c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:13.856847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2874d7b-0362-42cf-90fe-ee0f8a361903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.857470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2874d7b-0362-42cf-90fe-ee0f8a361903 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.857773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2874d7b-0362-42cf-90fe-ee0f8a361903 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.858056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2874d7b-0362-42cf-90fe-ee0f8a361903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.874554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cba4f-8952-4ec5-bb1a-91061ff298b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:13.875167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cba4f-8952-4ec5-bb1a-91061ff298b2 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:13.875446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cba4f-8952-4ec5-bb1a-91061ff298b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:13.875717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23cba4f-8952-4ec5-bb1a-91061ff298b2 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:13.876377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cba4f-8952-4ec5-bb1a-91061ff298b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.877857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23cba4f-8952-4ec5-bb1a-91061ff298b2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.899396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 43/88] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:57:13 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:13.900210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cba4f-8952-4ec5-bb1a-91061ff298b2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:13.931397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2874d7b-0362-42cf-90fe-ee0f8a361903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:13.933367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2874d7b-0362-42cf-90fe-ee0f8a361903 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:13.988027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:14.022769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23cba4f-8952-4ec5-bb1a-91061ff298b2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:14.043702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:14.054088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 46/89] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:14.055867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2874d7b-0362-42cf-90fe-ee0f8a361903 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:14.060174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942af3e-fc3c-442e-a6b2-1acb07244408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:14.060902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942af3e-fc3c-442e-a6b2-1acb07244408 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:14.061188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942af3e-fc3c-442e-a6b2-1acb07244408 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:14.061458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942af3e-fc3c-442e-a6b2-1acb07244408 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:14.138808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942af3e-fc3c-442e-a6b2-1acb07244408 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:14.140305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942af3e-fc3c-442e-a6b2-1acb07244408 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:14.176068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2874d7b-0362-42cf-90fe-ee0f8a361903 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:14.223988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfe296-bcca-493f-aff0-aff1b077040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:14.224667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfe296-bcca-493f-aff0-aff1b077040e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:14.224945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfe296-bcca-493f-aff0-aff1b077040e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:14.225208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcfe296-bcca-493f-aff0-aff1b077040e None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:14.226474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:14.227258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfe296-bcca-493f-aff0-aff1b077040e None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:14.228806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcfe296-bcca-493f-aff0-aff1b077040e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:14.230818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 47/90] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:14 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:14.231645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfe296-bcca-493f-aff0-aff1b077040e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:14.264302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 44/91] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:14.265221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942af3e-fc3c-442e-a6b2-1acb07244408 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:14.306662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:14.344121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcfe296-bcca-493f-aff0-aff1b077040e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:14.382770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:14.383680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:14.384122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:14.384559 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:14.385384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942af3e-fc3c-442e-a6b2-1acb07244408 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:14.385569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=d9df1c54939a4373af874595595358c5, user.id=e92af64dbb734bd49c69c6f84550ccd3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:14.386095 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:14.387894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:14.393897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 48/92] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:14 2022] GET /identity/v3/role_assignments?user.id=e92af64dbb734bd49c69c6f84550ccd3&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:14.394978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:14.486441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163dcb8-2009-4863-9445-305f411e8dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:14.487495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163dcb8-2009-4863-9445-305f411e8dcf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:14.487784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163dcb8-2009-4863-9445-305f411e8dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:14.488047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1163dcb8-2009-4863-9445-305f411e8dcf None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:14.491044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163dcb8-2009-4863-9445-305f411e8dcf None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:14.492580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1163dcb8-2009-4863-9445-305f411e8dcf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:14.494594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 45/93] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:14 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:14.495205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163dcb8-2009-4863-9445-305f411e8dcf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:14.519322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422bd5f-112c-4806-a44f-ecf49fac246b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:14.555964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:14.559806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:14.560674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:14.561150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:14.561556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:14.562876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] RBAC: Adding query filter params (role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=d9df1c54939a4373af874595595358c5, user.id=66a9447c53c74350bc5f1a7039895a77) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:14.563358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:14.565046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:14.573260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 49/94] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:14 2022] GET /identity/v3/role_assignments?user.id=66a9447c53c74350bc5f1a7039895a77&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=7b4a508ded224dfc812c05d174d41511 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:14.574597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:14.604973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:14.705744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2254d3-2602-443b-a697-f436431ed7f6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:14.733546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1163dcb8-2009-4863-9445-305f411e8dcf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:14.782940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b19a2a-b9b1-44f3-9788-fb41c0e4720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:14.783612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b19a2a-b9b1-44f3-9788-fb41c0e4720d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:14.783893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b19a2a-b9b1-44f3-9788-fb41c0e4720d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:14.784165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b19a2a-b9b1-44f3-9788-fb41c0e4720d None admin] PATH_INFO: `/v3/projects/35177400b30e4b81bcbd08ec003d5756` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:14.788163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b19a2a-b9b1-44f3-9788-fb41c0e4720d None admin] RBAC: Authorizing `identity:get_project(project_id=35177400b30e4b81bcbd08ec003d5756)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:14.789814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b19a2a-b9b1-44f3-9788-fb41c0e4720d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:14.792057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 46/95] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:14 2022] GET /identity/v3/projects/35177400b30e4b81bcbd08ec003d5756 => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:14.798081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b19a2a-b9b1-44f3-9788-fb41c0e4720d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:14.803447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e130e90e-837d-4973-8e7c-bbfca96bed16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:14.804837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e90e-837d-4973-8e7c-bbfca96bed16 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:14.805307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e90e-837d-4973-8e7c-bbfca96bed16 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:14.805890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e130e90e-837d-4973-8e7c-bbfca96bed16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:14.806823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130e90e-837d-4973-8e7c-bbfca96bed16 None admin] RBAC: Adding query filter params (role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=d9df1c54939a4373af874595595358c5, user.id=66a9447c53c74350bc5f1a7039895a77) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:14.807341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130e90e-837d-4973-8e7c-bbfca96bed16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:14.808991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e130e90e-837d-4973-8e7c-bbfca96bed16 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:14.819062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 50/96] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:14 2022] GET /identity/v3/role_assignments?user.id=66a9447c53c74350bc5f1a7039895a77&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=5491e6c9891843b8979a054e900d2e34 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:14.948813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b19a2a-b9b1-44f3-9788-fb41c0e4720d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:14.961393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:14.994784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae671a3b-df63-44c2-94d2-3dc8ce5e8c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:14.995778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae671a3b-df63-44c2-94d2-3dc8ce5e8c78 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:14.996316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae671a3b-df63-44c2-94d2-3dc8ce5e8c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:14.996778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae671a3b-df63-44c2-94d2-3dc8ce5e8c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:14.997736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae671a3b-df63-44c2-94d2-3dc8ce5e8c78 None admin] RBAC: Adding query filter params (role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=35177400b30e4b81bcbd08ec003d5756, user.id=66a9447c53c74350bc5f1a7039895a77) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:14.998265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae671a3b-df63-44c2-94d2-3dc8ce5e8c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:14.999939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae671a3b-df63-44c2-94d2-3dc8ce5e8c78 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:15.006918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 47/97] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:14 2022] GET /identity/v3/role_assignments?user.id=66a9447c53c74350bc5f1a7039895a77&scope.project.id=35177400b30e4b81bcbd08ec003d5756&role.id=7b4a508ded224dfc812c05d174d41511 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:17.157666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8aa3-8ce7-41c6-8599-1fde5f75445e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:17.158232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8aa3-8ce7-41c6-8599-1fde5f75445e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:17.158232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8aa3-8ce7-41c6-8599-1fde5f75445e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:17.159771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 51/98] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:17.165092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db784f-5d9f-4f2b-b65f-6a9060039553 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:17.165493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db784f-5d9f-4f2b-b65f-6a9060039553 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:17.165702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53db784f-5d9f-4f2b-b65f-6a9060039553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:17.171932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b4bc6-0e8f-4c80-8a68-cc92766ec23a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:17.172220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b4bc6-0e8f-4c80-8a68-cc92766ec23a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:17.172496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169b4bc6-0e8f-4c80-8a68-cc92766ec23a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:17.174091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 52/99] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:17.180703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b69c4-d32e-4970-9638-9d81335c2c17 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:17.180988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b69c4-d32e-4970-9638-9d81335c2c17 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:17.181265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b69c4-d32e-4970-9638-9d81335c2c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:17.207415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-53db784f-5d9f-4f2b-b65f-6a9060039553 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:17.208701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-53db784f-5d9f-4f2b-b65f-6a9060039553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:17.211764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-432b69c4-d32e-4970-9638-9d81335c2c17 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:17.212932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-432b69c4-d32e-4970-9638-9d81335c2c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:17.327048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:17.327232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 48/100] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:17.347001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 53/101] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:17.579623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdc3a6-5901-4ea1-9776-4c07943f298f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:17.579905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdc3a6-5901-4ea1-9776-4c07943f298f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:17.580425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bdc3a6-5901-4ea1-9776-4c07943f298f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:17.582030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 49/102] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:17.587603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274e016-6885-41d3-900a-06852b1c101f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:17.588137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274e016-6885-41d3-900a-06852b1c101f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:17.588599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8274e016-6885-41d3-900a-06852b1c101f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:17.607499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8274e016-6885-41d3-900a-06852b1c101f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:17.609043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8274e016-6885-41d3-900a-06852b1c101f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:17.672650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdc3a6-5901-4ea1-9776-4c07943f298f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:17.709705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 54/103] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:17.711396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983e503-ca4f-4be0-8546-3304aa9f70c8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:17.711688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983e503-ca4f-4be0-8546-3304aa9f70c8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:17.711968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b983e503-ca4f-4be0-8546-3304aa9f70c8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:17.713895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 55/104] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:17.719364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62641c-4bbf-4be2-9631-8c460238556e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:17.719652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62641c-4bbf-4be2-9631-8c460238556e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:17.720091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa62641c-4bbf-4be2-9631-8c460238556e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:17.741003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fa62641c-4bbf-4be2-9631-8c460238556e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:17.742455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fa62641c-4bbf-4be2-9631-8c460238556e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:17.842991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 56/105] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:17.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62641c-4bbf-4be2-9631-8c460238556e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:17.914015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:17.924722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bdc3a6-5901-4ea1-9776-4c07943f298f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:17.945699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:18.003670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa62641c-4bbf-4be2-9631-8c460238556e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:18.036036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a8f313-85d7-4e3d-9e58-c2ce5f12e358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.036749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8f313-85d7-4e3d-9e58-c2ce5f12e358 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.037025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8f313-85d7-4e3d-9e58-c2ce5f12e358 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.037296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a8f313-85d7-4e3d-9e58-c2ce5f12e358 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.041463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434f0ee8-a813-4d2d-948d-0ca92076e14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.042163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0ee8-a813-4d2d-948d-0ca92076e14d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.042328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8f313-85d7-4e3d-9e58-c2ce5f12e358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:18.042442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0ee8-a813-4d2d-948d-0ca92076e14d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.042879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434f0ee8-a813-4d2d-948d-0ca92076e14d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.043766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a8f313-85d7-4e3d-9e58-c2ce5f12e358 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:18.045417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 50/106] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:18.046714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380eafa-7104-49fe-afd5-50dab0dfd934 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.047131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380eafa-7104-49fe-afd5-50dab0dfd934 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.047535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380eafa-7104-49fe-afd5-50dab0dfd934 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.048854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 51/107] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:18.054750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380eafa-7104-49fe-afd5-50dab0dfd934 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.059190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434f0ee8-a813-4d2d-948d-0ca92076e14d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-434f0ee8-a813-4d2d-948d-0ca92076e14d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:18.065703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:18.066720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:18.067552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:18.068990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 57/108] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:18.071393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a092d6-8d77-485b-82c2-ed521380cc5d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.071877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a092d6-8d77-485b-82c2-ed521380cc5d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.072370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a092d6-8d77-485b-82c2-ed521380cc5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.096205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-39a092d6-8d77-485b-82c2-ed521380cc5d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:18.098614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-39a092d6-8d77-485b-82c2-ed521380cc5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:18.174179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380eafa-7104-49fe-afd5-50dab0dfd934 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:18.197802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 58/109] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:18.199143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a092d6-8d77-485b-82c2-ed521380cc5d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.211217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc717a2c-bb98-449c-9fbf-4cfc7d56fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.211908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc717a2c-bb98-449c-9fbf-4cfc7d56fd8d None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.212233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc717a2c-bb98-449c-9fbf-4cfc7d56fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.212505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc717a2c-bb98-449c-9fbf-4cfc7d56fd8d None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.213230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc717a2c-bb98-449c-9fbf-4cfc7d56fd8d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:18.214840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc717a2c-bb98-449c-9fbf-4cfc7d56fd8d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:18.245936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 52/110] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:57:18 2022] POST /identity/v3/projects => generated 302 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:57:18.246994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc717a2c-bb98-449c-9fbf-4cfc7d56fd8d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.258105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:18.301502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:18.384294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a092d6-8d77-485b-82c2-ed521380cc5d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:18.439464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc717a2c-bb98-449c-9fbf-4cfc7d56fd8d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:18.453621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b6139-4f42-4536-bc92-1f6a89e78e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.454391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b6139-4f42-4536-bc92-1f6a89e78e48 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.454614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b6139-4f42-4536-bc92-1f6a89e78e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.454888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b6139-4f42-4536-bc92-1f6a89e78e48 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.467463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b6139-4f42-4536-bc92-1f6a89e78e48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0d8b6139-4f42-4536-bc92-1f6a89e78e48 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:18.472425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:18.473327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:18.474165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 59/111] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:18.474766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b6139-4f42-4536-bc92-1f6a89e78e48 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.486452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569f7aa-bc68-4880-96dc-000ef007dedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.490632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569f7aa-bc68-4880-96dc-000ef007dedc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.490917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569f7aa-bc68-4880-96dc-000ef007dedc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.491189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569f7aa-bc68-4880-96dc-000ef007dedc None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.491881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569f7aa-bc68-4880-96dc-000ef007dedc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:18.492188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569f7aa-bc68-4880-96dc-000ef007dedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:18.493593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d569f7aa-bc68-4880-96dc-000ef007dedc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:18.504453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 53/112] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:18.505913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070150e3-efea-483c-8556-6740fd7c9603 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.506528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070150e3-efea-483c-8556-6740fd7c9603 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.506528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070150e3-efea-483c-8556-6740fd7c9603 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.507834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 54/113] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:18.508589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070150e3-efea-483c-8556-6740fd7c9603 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.592826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:18.596822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b6139-4f42-4536-bc92-1f6a89e78e48 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:18.604045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:18.626522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070150e3-efea-483c-8556-6740fd7c9603 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:18.635087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b5d2-4788-48b3-a9c6-cfcbc118b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.636005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b5d2-4788-48b3-a9c6-cfcbc118b6bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.636486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b5d2-4788-48b3-a9c6-cfcbc118b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.636958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec8b5d2-4788-48b3-a9c6-cfcbc118b6bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.648261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec8b5d2-4788-48b3-a9c6-cfcbc118b6bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cec8b5d2-4788-48b3-a9c6-cfcbc118b6bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:18.653406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:18.654336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:18.655211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:18.657109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 60/114] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:18.658037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b5d2-4788-48b3-a9c6-cfcbc118b6bf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.664742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.665373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.665708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.665985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.666667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:18.666971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:18.668391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:18.678225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 55/115] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:18.679040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.775925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec8b5d2-4788-48b3-a9c6-cfcbc118b6bf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:18.784848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe037e4-6800-4d7b-a526-6ae31e3ff53b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:18.814162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a40c27e-8382-448a-97cf-b602068f702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.814880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40c27e-8382-448a-97cf-b602068f702c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.815187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40c27e-8382-448a-97cf-b602068f702c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.815462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a40c27e-8382-448a-97cf-b602068f702c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.821105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d692a-3547-4cb9-8c2f-381b1defbb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:18.821951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d692a-3547-4cb9-8c2f-381b1defbb38 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.822284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d692a-3547-4cb9-8c2f-381b1defbb38 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.822560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d692a-3547-4cb9-8c2f-381b1defbb38 None admin] PATH_INFO: `/v3/users/249d095f4c484e1ab620fb17e16c8b33` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.827306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a40c27e-8382-448a-97cf-b602068f702c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6a40c27e-8382-448a-97cf-b602068f702c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:18.832548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:18.833394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:18.834597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 61/116] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:18.837228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaad021-0a0a-4ed2-8f8f-99676c05bf19 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:18.837924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaad021-0a0a-4ed2-8f8f-99676c05bf19 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:18.838337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaad021-0a0a-4ed2-8f8f-99676c05bf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:18.839157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d692a-3547-4cb9-8c2f-381b1defbb38 None admin] RBAC: Authorizing `identity:get_user(user_id=249d095f4c484e1ab620fb17e16c8b33)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:18.840610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d692a-3547-4cb9-8c2f-381b1defbb38 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:18.842303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 56/117] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:18 2022] GET /identity/v3/users/249d095f4c484e1ab620fb17e16c8b33 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:18.843175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d692a-3547-4cb9-8c2f-381b1defbb38 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.849441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:18.863488 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-efaad021-0a0a-4ed2-8f8f-99676c05bf19 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:18.864736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-efaad021-0a0a-4ed2-8f8f-99676c05bf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:18.879768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:18.948087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 62/118] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:18.948761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaad021-0a0a-4ed2-8f8f-99676c05bf19 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:18.955776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d692a-3547-4cb9-8c2f-381b1defbb38 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.003218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.004470 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.005417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.006155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.007235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:19.007743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.009944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.022141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 57/119] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:19.026711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.052735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaad021-0a0a-4ed2-8f8f-99676c05bf19 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.087930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78b47f-53a0-4b82-9651-36624e7a1a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.088863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78b47f-53a0-4b82-9651-36624e7a1a53 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.089310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78b47f-53a0-4b82-9651-36624e7a1a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.089784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab78b47f-53a0-4b82-9651-36624e7a1a53 None admin] PATH_INFO: `/v3/users/e92af64dbb734bd49c69c6f84550ccd3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.091263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78b47f-53a0-4b82-9651-36624e7a1a53 None admin] RBAC: Authorizing `identity:get_user(user_id=e92af64dbb734bd49c69c6f84550ccd3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.092909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab78b47f-53a0-4b82-9651-36624e7a1a53 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.094748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 63/120] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:18 2022] GET /identity/v3/users/e92af64dbb734bd49c69c6f84550ccd3 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:19.095675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78b47f-53a0-4b82-9651-36624e7a1a53 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.106316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:19.115740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:19.162636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a204534-eb45-4d99-b3a3-aba3594be9b8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.203062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.204476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.204969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.205428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.206325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:19.206843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.208584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.217125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab78b47f-53a0-4b82-9651-36624e7a1a53 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.219269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 58/121] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:19.220388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.261304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae968c-2909-47f9-a97c-4d5457f90650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.261980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae968c-2909-47f9-a97c-4d5457f90650 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.262262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae968c-2909-47f9-a97c-4d5457f90650 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.262538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae968c-2909-47f9-a97c-4d5457f90650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.329758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae968c-2909-47f9-a97c-4d5457f90650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.330676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512434-17d3-4e09-b4a4-12b37ad2245b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.331255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae968c-2909-47f9-a97c-4d5457f90650 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.351134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:19.356968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:19.368828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b4fb-7c1b-46cf-9288-7a95827fd219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.369479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4fb-7c1b-46cf-9288-7a95827fd219 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.369794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4fb-7c1b-46cf-9288-7a95827fd219 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.370073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a71b4fb-7c1b-46cf-9288-7a95827fd219 None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.371210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b4fb-7c1b-46cf-9288-7a95827fd219 None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.372804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a71b4fb-7c1b-46cf-9288-7a95827fd219 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.379421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 59/122] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:19.380322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b4fb-7c1b-46cf-9288-7a95827fd219 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.433600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 64/123] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:19.434325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae968c-2909-47f9-a97c-4d5457f90650 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.491358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a71b4fb-7c1b-46cf-9288-7a95827fd219 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.528326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476904-5a36-4130-8e4f-8c9e2a98b064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.528985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476904-5a36-4130-8e4f-8c9e2a98b064 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.529258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476904-5a36-4130-8e4f-8c9e2a98b064 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.529526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd476904-5a36-4130-8e4f-8c9e2a98b064 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.530915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd476904-5a36-4130-8e4f-8c9e2a98b064 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.532122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd476904-5a36-4130-8e4f-8c9e2a98b064 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.536177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 60/124] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:19.536493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476904-5a36-4130-8e4f-8c9e2a98b064 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.541589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae968c-2909-47f9-a97c-4d5457f90650 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.560408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:19.578992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa590c-e46f-4ab4-9a4f-77dcd4814f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.579640 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa590c-e46f-4ab4-9a4f-77dcd4814f8c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.579921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa590c-e46f-4ab4-9a4f-77dcd4814f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.580195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa590c-e46f-4ab4-9a4f-77dcd4814f8c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.592191 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa590c-e46f-4ab4-9a4f-77dcd4814f8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-93fa590c-e46f-4ab4-9a4f-77dcd4814f8c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:19.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:19.598010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:19.598755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 65/125] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:19.599442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa590c-e46f-4ab4-9a4f-77dcd4814f8c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.614284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:19.658881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd476904-5a36-4130-8e4f-8c9e2a98b064 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.697810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd96ca-5e1a-4137-9669-ccdbc4367710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.698451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd96ca-5e1a-4137-9669-ccdbc4367710 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.698725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd96ca-5e1a-4137-9669-ccdbc4367710 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.698990 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd96ca-5e1a-4137-9669-ccdbc4367710 None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.699080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa590c-e46f-4ab4-9a4f-77dcd4814f8c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.700117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd96ca-5e1a-4137-9669-ccdbc4367710 None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.702401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd96ca-5e1a-4137-9669-ccdbc4367710 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.704278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 61/126] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:19.705207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd96ca-5e1a-4137-9669-ccdbc4367710 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.737144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80196531-8695-4b6d-80ef-539fbc10b240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.737822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80196531-8695-4b6d-80ef-539fbc10b240 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.738369 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80196531-8695-4b6d-80ef-539fbc10b240 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.738639 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80196531-8695-4b6d-80ef-539fbc10b240 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.739821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80196531-8695-4b6d-80ef-539fbc10b240 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.741288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80196531-8695-4b6d-80ef-539fbc10b240 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.743224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 66/127] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:19.743703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80196531-8695-4b6d-80ef-539fbc10b240 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.799726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:19.817026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd96ca-5e1a-4137-9669-ccdbc4367710 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.846650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:19.853464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1470a-6d33-49a7-8c44-6c82774d0c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.854306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1470a-6d33-49a7-8c44-6c82774d0c74 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.854820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1470a-6d33-49a7-8c44-6c82774d0c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.855230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1470a-6d33-49a7-8c44-6c82774d0c74 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.855928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80196531-8695-4b6d-80ef-539fbc10b240 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:19.856644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1470a-6d33-49a7-8c44-6c82774d0c74 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.858263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1470a-6d33-49a7-8c44-6c82774d0c74 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.865171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 62/128] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:19.866192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1470a-6d33-49a7-8c44-6c82774d0c74 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.891865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ebf2e-ad54-4737-ab26-e4fc7aa41121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:19.892539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ebf2e-ad54-4737-ab26-e4fc7aa41121 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:19.892822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ebf2e-ad54-4737-ab26-e4fc7aa41121 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:19.893095 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ebf2e-ad54-4737-ab26-e4fc7aa41121 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5/users/e92af64dbb734bd49c69c6f84550ccd3/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:19.895049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ebf2e-ad54-4737-ab26-e4fc7aa41121 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9df1c54939a4373af874595595358c5, user_id=e92af64dbb734bd49c69c6f84550ccd3, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:19.896505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9ebf2e-ad54-4737-ab26-e4fc7aa41121 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:19.913571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 67/129] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:19 2022] PUT /identity/v3/projects/d9df1c54939a4373af874595595358c5/users/e92af64dbb734bd49c69c6f84550ccd3/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:19.915438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ebf2e-ad54-4737-ab26-e4fc7aa41121 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:19.989731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1470a-6d33-49a7-8c44-6c82774d0c74 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:20.028144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:20.028687 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.029013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.029341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.030108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:20.030413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:20.031809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:20.041703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 63/130] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:20.042575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:20.067687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ebf2e-ad54-4737-ab26-e4fc7aa41121 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:20.076607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:20.107960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:20.114563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386e854-41f5-46d9-981e-84adfb16a975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:20.115523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386e854-41f5-46d9-981e-84adfb16a975 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.116075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386e854-41f5-46d9-981e-84adfb16a975 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.116536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386e854-41f5-46d9-981e-84adfb16a975 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.117948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386e854-41f5-46d9-981e-84adfb16a975 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:20.119672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a386e854-41f5-46d9-981e-84adfb16a975 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:20.121456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 68/131] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:19 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:20.122534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386e854-41f5-46d9-981e-84adfb16a975 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:20.155306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3bc08-a549-49f3-8cfc-098796484128 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:20.191813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601723e4-8c5e-483c-9c98-4ed9e7043ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:20.192488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601723e4-8c5e-483c-9c98-4ed9e7043ab4 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.192797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601723e4-8c5e-483c-9c98-4ed9e7043ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.193069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601723e4-8c5e-483c-9c98-4ed9e7043ab4 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5/users/66a9447c53c74350bc5f1a7039895a77/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.195051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601723e4-8c5e-483c-9c98-4ed9e7043ab4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9df1c54939a4373af874595595358c5, user_id=66a9447c53c74350bc5f1a7039895a77, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:20.196591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601723e4-8c5e-483c-9c98-4ed9e7043ab4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:20.213428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 64/132] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:20 2022] PUT /identity/v3/projects/d9df1c54939a4373af874595595358c5/users/66a9447c53c74350bc5f1a7039895a77/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:20.214316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601723e4-8c5e-483c-9c98-4ed9e7043ab4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:20.249266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386e854-41f5-46d9-981e-84adfb16a975 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:20.292022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:20.293117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.293591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.294118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.295239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] RBAC: Adding query filter params (role.id=2e8c5a373cb5440d91307a8c8ab3186d, scope.project.id=d9df1c54939a4373af874595595358c5, user.id=249d095f4c484e1ab620fb17e16c8b33) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:20.295794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:20.297579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:20.304880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 69/133] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:20 2022] GET /identity/v3/role_assignments?user.id=249d095f4c484e1ab620fb17e16c8b33&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=2e8c5a373cb5440d91307a8c8ab3186d => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:20.306318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:20.335456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601723e4-8c5e-483c-9c98-4ed9e7043ab4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:20.353361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:20.361466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:20.380228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e94bb-bd41-40d3-8844-5635b5e66267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:20.380917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e94bb-bd41-40d3-8844-5635b5e66267 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.381202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e94bb-bd41-40d3-8844-5635b5e66267 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.381478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e94bb-bd41-40d3-8844-5635b5e66267 None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.382830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e94bb-bd41-40d3-8844-5635b5e66267 None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:20.384320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e94bb-bd41-40d3-8844-5635b5e66267 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:20.389251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 65/134] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:20 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:20.392607 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e94bb-bd41-40d3-8844-5635b5e66267 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:20.460116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46754312-ac9e-41a6-bfe5-963142767571 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:20.522380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e94bb-bd41-40d3-8844-5635b5e66267 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:20.535215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d3f78e-d61c-4a98-b89f-b774cff17dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:20.535963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f78e-d61c-4a98-b89f-b774cff17dec None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.536244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f78e-d61c-4a98-b89f-b774cff17dec None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.536770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d3f78e-d61c-4a98-b89f-b774cff17dec None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5/users/66a9447c53c74350bc5f1a7039895a77/roles/5491e6c9891843b8979a054e900d2e34` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.539104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3f78e-d61c-4a98-b89f-b774cff17dec None admin] RBAC: Authorizing `identity:create_grant(project_id=d9df1c54939a4373af874595595358c5, user_id=66a9447c53c74350bc5f1a7039895a77, role_id=5491e6c9891843b8979a054e900d2e34)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:20.540611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d3f78e-d61c-4a98-b89f-b774cff17dec None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:20.559998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7b78-550a-495f-b530-666f4d9c5537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:20.560631 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7b78-550a-495f-b530-666f4d9c5537 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.560959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7b78-550a-495f-b530-666f4d9c5537 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.561238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4d7b78-550a-495f-b530-666f4d9c5537 None admin] PATH_INFO: `/v3/projects/35177400b30e4b81bcbd08ec003d5756` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.562617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7b78-550a-495f-b530-666f4d9c5537 None admin] RBAC: Authorizing `identity:get_project(project_id=35177400b30e4b81bcbd08ec003d5756)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:20.564159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4d7b78-550a-495f-b530-666f4d9c5537 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:20.566022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 66/135] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:20 2022] GET /identity/v3/projects/35177400b30e4b81bcbd08ec003d5756 => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:20.567819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 70/136] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:20 2022] PUT /identity/v3/projects/d9df1c54939a4373af874595595358c5/users/66a9447c53c74350bc5f1a7039895a77/roles/5491e6c9891843b8979a054e900d2e34 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:20.572664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7b78-550a-495f-b530-666f4d9c5537 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:20.672018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:20.723646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4d7b78-550a-495f-b530-666f4d9c5537 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:20.762367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd92d2-abf4-4556-a11a-4c8a06122b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:20.763072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd92d2-abf4-4556-a11a-4c8a06122b4c None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.763356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd92d2-abf4-4556-a11a-4c8a06122b4c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.763628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd92d2-abf4-4556-a11a-4c8a06122b4c None admin] PATH_INFO: `/v3/projects/35177400b30e4b81bcbd08ec003d5756/users/66a9447c53c74350bc5f1a7039895a77/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.765688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd92d2-abf4-4556-a11a-4c8a06122b4c None admin] RBAC: Authorizing `identity:create_grant(project_id=35177400b30e4b81bcbd08ec003d5756, user_id=66a9447c53c74350bc5f1a7039895a77, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:20.767225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acd92d2-abf4-4556-a11a-4c8a06122b4c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:20.792490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 67/137] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:20 2022] PUT /identity/v3/projects/35177400b30e4b81bcbd08ec003d5756/users/66a9447c53c74350bc5f1a7039895a77/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:20.822719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff7a9-c554-44d1-af4f-908d40a60268 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.823011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff7a9-c554-44d1-af4f-908d40a60268 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.823182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ff7a9-c554-44d1-af4f-908d40a60268 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.824649 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 71/138] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:20.830958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c1ac2-9345-4faa-99f0-238df47c031f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:20.831242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c1ac2-9345-4faa-99f0-238df47c031f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:20.831525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c1ac2-9345-4faa-99f0-238df47c031f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:20.856498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-347c1ac2-9345-4faa-99f0-238df47c031f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:20.857803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-347c1ac2-9345-4faa-99f0-238df47c031f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:20.940984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 68/139] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:21.355970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ff7a9-c554-44d1-af4f-908d40a60268 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:21.478127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:21.483889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ff7a9-c554-44d1-af4f-908d40a60268 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:21.544236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b6eb3-e69d-44a4-83a0-2be2006d6ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:21.544971 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b6eb3-e69d-44a4-83a0-2be2006d6ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:21.545255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b6eb3-e69d-44a4-83a0-2be2006d6ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:21.545527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b6eb3-e69d-44a4-83a0-2be2006d6ee6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:21.546732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b6eb3-e69d-44a4-83a0-2be2006d6ee6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:21.548231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783b6eb3-e69d-44a4-83a0-2be2006d6ee6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:21.550215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 72/140] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:21.557992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c1ac2-9345-4faa-99f0-238df47c031f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:21.628621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:21.687316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c1ac2-9345-4faa-99f0-238df47c031f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:21.737906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d040a-2919-49d5-ba23-0f4c19660ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:21.738532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d040a-2919-49d5-ba23-0f4c19660ab7 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:21.738802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d040a-2919-49d5-ba23-0f4c19660ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:21.739066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d040a-2919-49d5-ba23-0f4c19660ab7 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:21.739819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d040a-2919-49d5-ba23-0f4c19660ab7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:21.741272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6d040a-2919-49d5-ba23-0f4c19660ab7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:21.773189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 69/141] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:57:21 2022] POST /identity/v3/users => generated 283 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:22.599570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f8ab-9cd4-41f4-b7b2-0b7f048d188c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:22.600540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f8ab-9cd4-41f4-b7b2-0b7f048d188c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:22.601035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f8ab-9cd4-41f4-b7b2-0b7f048d188c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:22.602900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 73/142] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:22.610703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06a131-98d7-492e-8edd-376db94612f9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:22.611151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06a131-98d7-492e-8edd-376db94612f9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:22.611336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06a131-98d7-492e-8edd-376db94612f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:22.632180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0b06a131-98d7-492e-8edd-376db94612f9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:22.633360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b06a131-98d7-492e-8edd-376db94612f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:22.706055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 70/143] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:22.999047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505f8ab-9cd4-41f4-b7b2-0b7f048d188c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:23.122617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505f8ab-9cd4-41f4-b7b2-0b7f048d188c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:23.164735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c2eac-6abb-4665-9094-84e0e196cfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:23.165385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c2eac-6abb-4665-9094-84e0e196cfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:23.165691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c2eac-6abb-4665-9094-84e0e196cfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:23.165962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909c2eac-6abb-4665-9094-84e0e196cfc9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:23.180387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909c2eac-6abb-4665-9094-84e0e196cfc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-909c2eac-6abb-4665-9094-84e0e196cfc9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:23.185112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:23.185951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:23.186699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 74/144] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:23.192732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06a131-98d7-492e-8edd-376db94612f9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:23.226706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:23.298733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06a131-98d7-492e-8edd-376db94612f9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:23.334425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1848de5-6c16-4188-9780-65ac5b7cada8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:23.335050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848de5-6c16-4188-9780-65ac5b7cada8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:23.335324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848de5-6c16-4188-9780-65ac5b7cada8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:23.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1848de5-6c16-4188-9780-65ac5b7cada8 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:23.336272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1848de5-6c16-4188-9780-65ac5b7cada8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:23.336570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1848de5-6c16-4188-9780-65ac5b7cada8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:23.337996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1848de5-6c16-4188-9780-65ac5b7cada8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:23.350651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 71/145] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:23.353513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c2eac-6abb-4665-9094-84e0e196cfc9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:23.365485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84f422-e8a5-44ae-a332-9ef71c904ad7 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:23.365800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84f422-e8a5-44ae-a332-9ef71c904ad7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:23.366058 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84f422-e8a5-44ae-a332-9ef71c904ad7 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:23.367417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 72/146] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:23.373129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243193-fc07-43d2-a4de-8c0ad108d29d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:23.373412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243193-fc07-43d2-a4de-8c0ad108d29d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:23.373703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b243193-fc07-43d2-a4de-8c0ad108d29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:23.404338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:23.406920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2b243193-fc07-43d2-a4de-8c0ad108d29d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:23.408243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2b243193-fc07-43d2-a4de-8c0ad108d29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:23.489285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 73/147] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:23.491041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8e05b-5f72-4333-902e-889679ed38bb None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:23.491324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8e05b-5f72-4333-902e-889679ed38bb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:23.491595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a8e05b-5f72-4333-902e-889679ed38bb None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:23.493122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 74/148] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:23.500649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a145ee-0747-4002-bdd3-5f11941f7e3f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:23.501175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a145ee-0747-4002-bdd3-5f11941f7e3f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:23.501537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a145ee-0747-4002-bdd3-5f11941f7e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:23.528205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f0a145ee-0747-4002-bdd3-5f11941f7e3f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:23.529553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f0a145ee-0747-4002-bdd3-5f11941f7e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:23.546815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909c2eac-6abb-4665-9094-84e0e196cfc9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:23.551193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:23.598202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37220c4-71c3-4dac-9008-c404e60f3926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:23.599251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37220c4-71c3-4dac-9008-c404e60f3926 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:23.600541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37220c4-71c3-4dac-9008-c404e60f3926 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:23.602198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37220c4-71c3-4dac-9008-c404e60f3926 None admin] PATH_INFO: `/v3/users/e92af64dbb734bd49c69c6f84550ccd3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:23.603674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37220c4-71c3-4dac-9008-c404e60f3926 None admin] RBAC: Authorizing `identity:get_user(user_id=e92af64dbb734bd49c69c6f84550ccd3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:23.606585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37220c4-71c3-4dac-9008-c404e60f3926 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:23.609141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 75/149] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:23 2022] GET /identity/v3/users/e92af64dbb734bd49c69c6f84550ccd3 => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:23.618661 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37220c4-71c3-4dac-9008-c404e60f3926 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:23.644414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 75/150] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:23.724155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:23.736209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37220c4-71c3-4dac-9008-c404e60f3926 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:23.781238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1c495-9c7c-45bb-ba40-ac09d853b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:23.781887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1c495-9c7c-45bb-ba40-ac09d853b6ad None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:23.782161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1c495-9c7c-45bb-ba40-ac09d853b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:23.782430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f1c495-9c7c-45bb-ba40-ac09d853b6ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:23.863504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1c495-9c7c-45bb-ba40-ac09d853b6ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:23.864988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f1c495-9c7c-45bb-ba40-ac09d853b6ad None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:23.915535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a145ee-0747-4002-bdd3-5f11941f7e3f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:24.009315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:24.020326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 76/151] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:24.025573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1c495-9c7c-45bb-ba40-ac09d853b6ad None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:24.146690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a145ee-0747-4002-bdd3-5f11941f7e3f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:24.179987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f1c495-9c7c-45bb-ba40-ac09d853b6ad None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:24.231315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aeed4-65a3-4998-b9b9-d249f9613a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:24.231967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aeed4-65a3-4998-b9b9-d249f9613a25 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:24.232244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aeed4-65a3-4998-b9b9-d249f9613a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:24.232511 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04aeed4-65a3-4998-b9b9-d249f9613a25 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:24.233682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aeed4-65a3-4998-b9b9-d249f9613a25 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:24.235368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04aeed4-65a3-4998-b9b9-d249f9613a25 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:24.237761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 77/152] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:24 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:24.247072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aeed4-65a3-4998-b9b9-d249f9613a25 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:24.253347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e71b6-8a74-4677-bba7-291c58ca7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:24.254043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e71b6-8a74-4677-bba7-291c58ca7f69 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:24.254314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e71b6-8a74-4677-bba7-291c58ca7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:24.254582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e71b6-8a74-4677-bba7-291c58ca7f69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:24.286583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e71b6-8a74-4677-bba7-291c58ca7f69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-de4e71b6-8a74-4677-bba7-291c58ca7f69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:24.291436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:24.292293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:24.293071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 76/153] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:24.302693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e71b6-8a74-4677-bba7-291c58ca7f69 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:24.341719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:24.374004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:24.469551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04aeed4-65a3-4998-b9b9-d249f9613a25 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:24.538033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e71b6-8a74-4677-bba7-291c58ca7f69 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:24.574401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:24.575052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:24.575335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:24.575605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:24.576359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=d9df1c54939a4373af874595595358c5, user.id=e92af64dbb734bd49c69c6f84550ccd3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:24.576665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:24.578180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:24.595896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 78/154] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:24 2022] GET /identity/v3/role_assignments?user.id=e92af64dbb734bd49c69c6f84550ccd3&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 608 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:24.596799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:24.620339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:24.620695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:24.620871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:24.621135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:24.621838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:24.622145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:24.623560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:24.642326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 77/155] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:24.649489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:24.802591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995f096-86cd-4fe3-82e5-e748d1d23156 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:24.807839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:24.847817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:24.859754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339cd07-1ec9-4eea-a471-5f7940d3f5d1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:24.866023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e5e2f7-fedf-4050-86d2-b916ebb16bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:24.866726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e2f7-fedf-4050-86d2-b916ebb16bea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:24.867002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e2f7-fedf-4050-86d2-b916ebb16bea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:24.867270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e5e2f7-fedf-4050-86d2-b916ebb16bea None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:24.882583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e5e2f7-fedf-4050-86d2-b916ebb16bea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-31e5e2f7-fedf-4050-86d2-b916ebb16bea None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:24.887870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:24.888911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:24.889784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:24.894046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 79/156] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:24.895345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152098a-83d4-467c-80cd-eb1333076b25 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:24.895663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152098a-83d4-467c-80cd-eb1333076b25 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:24.895931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152098a-83d4-467c-80cd-eb1333076b25 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:24.897355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 80/157] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:24.899840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152098a-83d4-467c-80cd-eb1333076b25 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:24.955590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c1fa4-c219-4220-b283-c3eae5d731ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:24.956390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c1fa4-c219-4220-b283-c3eae5d731ed None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:24.956672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c1fa4-c219-4220-b283-c3eae5d731ed None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:24.956946 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c1fa4-c219-4220-b283-c3eae5d731ed None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:24.958337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c1fa4-c219-4220-b283-c3eae5d731ed None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:24.959801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88c1fa4-c219-4220-b283-c3eae5d731ed None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:24.961568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 78/158] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:24 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:24.967048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff116b2-5127-4802-9234-d0c49c1d508c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:24.967321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff116b2-5127-4802-9234-d0c49c1d508c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:24.967602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff116b2-5127-4802-9234-d0c49c1d508c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:25.014844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9ff116b2-5127-4802-9234-d0c49c1d508c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:25.016080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9ff116b2-5127-4802-9234-d0c49c1d508c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:25.095930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152098a-83d4-467c-80cd-eb1333076b25 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:25.164043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:25.164701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:25.164948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:25.165205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:25.166110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:25.166410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:25.167901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:25.179123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 79/159] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:25.180053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff116b2-5127-4802-9234-d0c49c1d508c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:25.186116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 81/160] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:25.192130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:25.270080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:25.300633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:25.335849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff116b2-5127-4802-9234-d0c49c1d508c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:25.360673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2be44-f54f-4ae8-8a6e-0f1665e2f42a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:25.392856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0177b-2198-4774-90c8-09d678284649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:25.393761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0177b-2198-4774-90c8-09d678284649 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:25.394911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0177b-2198-4774-90c8-09d678284649 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:25.395351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0177b-2198-4774-90c8-09d678284649 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:25.435336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91fafb0-3c96-4996-88f5-b2119215593f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:25.437066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fafb0-3c96-4996-88f5-b2119215593f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:25.438209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fafb0-3c96-4996-88f5-b2119215593f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:25.438786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91fafb0-3c96-4996-88f5-b2119215593f None admin] PATH_INFO: `/v3/users/249d095f4c484e1ab620fb17e16c8b33` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:25.441116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fafb0-3c96-4996-88f5-b2119215593f None admin] RBAC: Authorizing `identity:get_user(user_id=249d095f4c484e1ab620fb17e16c8b33)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:25.442957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91fafb0-3c96-4996-88f5-b2119215593f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:25.445494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 82/161] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:25 2022] GET /identity/v3/users/249d095f4c484e1ab620fb17e16c8b33 => generated 289 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:25.453811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1b37-691b-435a-85f0-fdfe2b88d886 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:25.455184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1b37-691b-435a-85f0-fdfe2b88d886 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:25.455758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afa1b37-691b-435a-85f0-fdfe2b88d886 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:25.458269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 83/162] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:25.461628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa1b37-691b-435a-85f0-fdfe2b88d886 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:25.552868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0177b-2198-4774-90c8-09d678284649 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:25.554879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0177b-2198-4774-90c8-09d678284649 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:25.653538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afa1b37-691b-435a-85f0-fdfe2b88d886 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:25.682004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:25.684316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:25.706098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b7dff-5d5b-4ac3-bcb7-0d38f9ff73d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:25.707005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b7dff-5d5b-4ac3-bcb7-0d38f9ff73d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:25.707433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b7dff-5d5b-4ac3-bcb7-0d38f9ff73d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:25.707850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b7dff-5d5b-4ac3-bcb7-0d38f9ff73d1 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:25.709166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b7dff-5d5b-4ac3-bcb7-0d38f9ff73d1 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:25.710827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12b7dff-5d5b-4ac3-bcb7-0d38f9ff73d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:25.713057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 84/163] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:25 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:25.714487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16d25-3a0f-47d9-a53c-759c535ea684 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:25.714982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16d25-3a0f-47d9-a53c-759c535ea684 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:25.715407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb16d25-3a0f-47d9-a53c-759c535ea684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:25.732761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 80/164] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:25.733682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0177b-2198-4774-90c8-09d678284649 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:25.738666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7fb16d25-3a0f-47d9-a53c-759c535ea684 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:25.739957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7fb16d25-3a0f-47d9-a53c-759c535ea684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:25.857911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 85/165] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:25.858890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb16d25-3a0f-47d9-a53c-759c535ea684 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:25.897817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0177b-2198-4774-90c8-09d678284649 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:25.961555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336254-5bee-4632-b7e5-bec9008d9284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:25.962152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336254-5bee-4632-b7e5-bec9008d9284 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:25.962424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336254-5bee-4632-b7e5-bec9008d9284 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:25.962688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f336254-5bee-4632-b7e5-bec9008d9284 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:25.985003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f336254-5bee-4632-b7e5-bec9008d9284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7f336254-5bee-4632-b7e5-bec9008d9284 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:25.991878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:25.992775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:25.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:25.996622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 81/166] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:25.997617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336254-5bee-4632-b7e5-bec9008d9284 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:26.040953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:26.046807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb16d25-3a0f-47d9-a53c-759c535ea684 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:26.076598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:26.125527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b28fcb-c7a5-4a45-8533-01e4223871ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:26.126237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28fcb-c7a5-4a45-8533-01e4223871ed None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:26.126509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28fcb-c7a5-4a45-8533-01e4223871ed None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:26.126772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b28fcb-c7a5-4a45-8533-01e4223871ed None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5/users/249d095f4c484e1ab620fb17e16c8b33/roles/2e8c5a373cb5440d91307a8c8ab3186d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:26.134955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b28fcb-c7a5-4a45-8533-01e4223871ed None admin] RBAC: Authorizing `identity:create_grant(project_id=d9df1c54939a4373af874595595358c5, user_id=249d095f4c484e1ab620fb17e16c8b33, role_id=2e8c5a373cb5440d91307a8c8ab3186d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:26.136422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b28fcb-c7a5-4a45-8533-01e4223871ed None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:26.140828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f336254-5bee-4632-b7e5-bec9008d9284 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:26.164225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 86/167] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:25 2022] PUT /identity/v3/projects/d9df1c54939a4373af874595595358c5/users/249d095f4c484e1ab620fb17e16c8b33/roles/2e8c5a373cb5440d91307a8c8ab3186d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:26.165129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b28fcb-c7a5-4a45-8533-01e4223871ed None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:26.196871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2b391-928c-4593-ba95-7367fe67eaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:26.197980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b391-928c-4593-ba95-7367fe67eaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:26.198442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b391-928c-4593-ba95-7367fe67eaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:26.198890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc2b391-928c-4593-ba95-7367fe67eaa2 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:26.201297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2b391-928c-4593-ba95-7367fe67eaa2 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:26.203103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc2b391-928c-4593-ba95-7367fe67eaa2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:26.205235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 82/168] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:25 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:26.212108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2b391-928c-4593-ba95-7367fe67eaa2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:26.352803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b28fcb-c7a5-4a45-8533-01e4223871ed None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:26.390153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc2b391-928c-4593-ba95-7367fe67eaa2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:26.398945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:26.424038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:26.424683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:26.424956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:26.425227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:26.425942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:26.426251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:26.427775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:26.445934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 87/169] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:26.446864 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:26.455449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:26.456102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:26.456369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:26.456645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:26.457388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] RBAC: Adding query filter params (role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=d9df1c54939a4373af874595595358c5, user.id=66a9447c53c74350bc5f1a7039895a77) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:26.457720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:26.459265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:26.472604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 83/170] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:26 2022] GET /identity/v3/role_assignments?user.id=66a9447c53c74350bc5f1a7039895a77&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=7b4a508ded224dfc812c05d174d41511 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:26.473453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:26.531407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:26.607967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903d153-2468-477f-ab86-d40f0b447c51 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:26.644754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43213f-cbb9-4c35-9a4e-bdfe3787227b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:26.645419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43213f-cbb9-4c35-9a4e-bdfe3787227b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:26.645765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43213f-cbb9-4c35-9a4e-bdfe3787227b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:26.646042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43213f-cbb9-4c35-9a4e-bdfe3787227b None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:26.647853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43213f-cbb9-4c35-9a4e-bdfe3787227b None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:26.649338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43213f-cbb9-4c35-9a4e-bdfe3787227b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:26.651210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 84/171] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:26 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:26.656886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43213f-cbb9-4c35-9a4e-bdfe3787227b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:26.668683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b726738-fe6e-4976-8f91-711cf96f5aca None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:26.696212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:26.748174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8484105b-2fda-44b6-b13e-7d388256c677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:26.748753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484105b-2fda-44b6-b13e-7d388256c677 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:26.749021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484105b-2fda-44b6-b13e-7d388256c677 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:26.749282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8484105b-2fda-44b6-b13e-7d388256c677 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:26.785140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8484105b-2fda-44b6-b13e-7d388256c677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8484105b-2fda-44b6-b13e-7d388256c677 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:26.790132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:26.791065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:26.791888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 88/172] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:26.793340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287932-2ae2-4c71-8f05-c1cc8575bd59 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:26.793340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287932-2ae2-4c71-8f05-c1cc8575bd59 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:26.793340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78287932-2ae2-4c71-8f05-c1cc8575bd59 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:26.803384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 89/173] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:26.804326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287932-2ae2-4c71-8f05-c1cc8575bd59 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:26.816066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43213f-cbb9-4c35-9a4e-bdfe3787227b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:26.887366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaa642f-f550-4e6c-a222-8c175f2baf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:26.887999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa642f-f550-4e6c-a222-8c175f2baf05 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:26.888269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa642f-f550-4e6c-a222-8c175f2baf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:26.888534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaa642f-f550-4e6c-a222-8c175f2baf05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:26.964871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa642f-f550-4e6c-a222-8c175f2baf05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:26.966371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaa642f-f550-4e6c-a222-8c175f2baf05 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:26.984233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:26.994214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78287932-2ae2-4c71-8f05-c1cc8575bd59 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:27.003352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:27.042693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7229e-ad6d-4209-842e-19846aea67d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:27.043382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7229e-ad6d-4209-842e-19846aea67d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.043819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7229e-ad6d-4209-842e-19846aea67d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.044265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7229e-ad6d-4209-842e-19846aea67d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.045897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7229e-ad6d-4209-842e-19846aea67d1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:27.046201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7229e-ad6d-4209-842e-19846aea67d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:27.047702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7229e-ad6d-4209-842e-19846aea67d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:27.058571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 90/174] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:27.059957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3245-67e2-4bf2-b375-1f38e1d3ef76 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.060236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3245-67e2-4bf2-b375-1f38e1d3ef76 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.060499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e3245-67e2-4bf2-b375-1f38e1d3ef76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.086897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d79e3245-67e2-4bf2-b375-1f38e1d3ef76 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:27.088348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d79e3245-67e2-4bf2-b375-1f38e1d3ef76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:27.102241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 85/175] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:27.105760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99af3b-8189-4e8d-b52c-86bd78c8724e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.106038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99af3b-8189-4e8d-b52c-86bd78c8724e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.106302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99af3b-8189-4e8d-b52c-86bd78c8724e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.107690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 86/176] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:27.108670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99af3b-8189-4e8d-b52c-86bd78c8724e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:27.177106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 91/177] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:27.182753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e3245-67e2-4bf2-b375-1f38e1d3ef76 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:27.233597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99af3b-8189-4e8d-b52c-86bd78c8724e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:27.271150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a82a243-fb7c-443d-9aff-e1bd0ef80c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:27.272261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a243-fb7c-443d-9aff-e1bd0ef80c89 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.272703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a243-fb7c-443d-9aff-e1bd0ef80c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.273147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a82a243-fb7c-443d-9aff-e1bd0ef80c89 None admin] PATH_INFO: `/v3/users/66a9447c53c74350bc5f1a7039895a77` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.274952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82a243-fb7c-443d-9aff-e1bd0ef80c89 None admin] RBAC: Authorizing `identity:get_user(user_id=66a9447c53c74350bc5f1a7039895a77)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:27.276601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a82a243-fb7c-443d-9aff-e1bd0ef80c89 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:27.278441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 87/178] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:27 2022] GET /identity/v3/users/66a9447c53c74350bc5f1a7039895a77 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:27.279888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69774-3d0b-420c-98b3-15413719cbc6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.280348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69774-3d0b-420c-98b3-15413719cbc6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.280828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d69774-3d0b-420c-98b3-15413719cbc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.287383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:27.302781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-51d69774-3d0b-420c-98b3-15413719cbc6 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:27.305059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51d69774-3d0b-420c-98b3-15413719cbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:27.343487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:27.358464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e3245-67e2-4bf2-b375-1f38e1d3ef76 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:27.387955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 88/179] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:27.389159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d69774-3d0b-420c-98b3-15413719cbc6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:27.435266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8acd1c5-de72-4fe8-a15b-13eb33ba6e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:27.435943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acd1c5-de72-4fe8-a15b-13eb33ba6e70 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.436222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acd1c5-de72-4fe8-a15b-13eb33ba6e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.436494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8acd1c5-de72-4fe8-a15b-13eb33ba6e70 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.442326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acd1c5-de72-4fe8-a15b-13eb33ba6e70 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:27.443852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8acd1c5-de72-4fe8-a15b-13eb33ba6e70 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:27.445514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 92/180] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:27 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:27.446714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69456b6-731f-498b-b9f4-3e5f8449b099 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.446993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69456b6-731f-498b-b9f4-3e5f8449b099 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.447258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69456b6-731f-498b-b9f4-3e5f8449b099 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.448460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 93/181] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:27.457570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea4f45-88ab-4803-bd71-d1b9c56d16bf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.457898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea4f45-88ab-4803-bd71-d1b9c56d16bf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.458215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ea4f45-88ab-4803-bd71-d1b9c56d16bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.494780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b8ea4f45-88ab-4803-bd71-d1b9c56d16bf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:27.496092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b8ea4f45-88ab-4803-bd71-d1b9c56d16bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:27.528033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d69774-3d0b-420c-98b3-15413719cbc6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:27.562736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:27.570989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837659-4806-4289-9dd6-84b24e03ba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:27.572194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837659-4806-4289-9dd6-84b24e03ba51 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.573585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837659-4806-4289-9dd6-84b24e03ba51 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.574507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71837659-4806-4289-9dd6-84b24e03ba51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.585767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 94/182] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:27.588833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea4f45-88ab-4803-bd71-d1b9c56d16bf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:27.706180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71837659-4806-4289-9dd6-84b24e03ba51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:27.707683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71837659-4806-4289-9dd6-84b24e03ba51 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:27.714669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:27.737378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ea4f45-88ab-4803-bd71-d1b9c56d16bf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:27.784544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:27.786559 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.787012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.787440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:27.788408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] RBAC: Adding query filter params (role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=d9df1c54939a4373af874595595358c5, user.id=66a9447c53c74350bc5f1a7039895a77) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:27.788884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:27.790506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:27.798757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 95/183] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:27 2022] GET /identity/v3/role_assignments?user.id=66a9447c53c74350bc5f1a7039895a77&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=5491e6c9891843b8979a054e900d2e34 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:27.799751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:27.844566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 89/184] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:27.845547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837659-4806-4289-9dd6-84b24e03ba51 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:27.897983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:27.935458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeaaa0e-553a-487e-b9dd-2e4edd7f1162 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:27.985474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71837659-4806-4289-9dd6-84b24e03ba51 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:27.986059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cec93-53ab-4952-8de5-e5c4432065c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:27.987918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cec93-53ab-4952-8de5-e5c4432065c6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:27.989428 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cec93-53ab-4952-8de5-e5c4432065c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:27.989918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cec93-53ab-4952-8de5-e5c4432065c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.008339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60cec93-53ab-4952-8de5-e5c4432065c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f60cec93-53ab-4952-8de5-e5c4432065c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:28.016130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:28.017058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:28.017886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:28.020250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 96/185] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:28.020936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cec93-53ab-4952-8de5-e5c4432065c6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.035979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc65ee-d342-4b2f-a1d0-21bd4793ce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.036793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc65ee-d342-4b2f-a1d0-21bd4793ce12 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.036952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:28.037205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc65ee-d342-4b2f-a1d0-21bd4793ce12 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.037563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabc65ee-d342-4b2f-a1d0-21bd4793ce12 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.051469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabc65ee-d342-4b2f-a1d0-21bd4793ce12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-fabc65ee-d342-4b2f-a1d0-21bd4793ce12 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:28.056493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:28.057382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:28.058190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 90/186] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:28.058911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc65ee-d342-4b2f-a1d0-21bd4793ce12 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.159610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cec93-53ab-4952-8de5-e5c4432065c6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.179603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:28.189675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabc65ee-d342-4b2f-a1d0-21bd4793ce12 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.195484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d40f3-fee7-43da-aab0-297cbad8f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.196144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d40f3-fee7-43da-aab0-297cbad8f8cf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.196420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d40f3-fee7-43da-aab0-297cbad8f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.196689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d40f3-fee7-43da-aab0-297cbad8f8cf None admin] PATH_INFO: `/v3/projects/35177400b30e4b81bcbd08ec003d5756` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.197861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d40f3-fee7-43da-aab0-297cbad8f8cf None admin] RBAC: Authorizing `identity:get_project(project_id=35177400b30e4b81bcbd08ec003d5756)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:28.199508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d40f3-fee7-43da-aab0-297cbad8f8cf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:28.201121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 97/187] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/projects/35177400b30e4b81bcbd08ec003d5756 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:28.201813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d40f3-fee7-43da-aab0-297cbad8f8cf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.230246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e994e1-6f9a-42d8-b9a7-04d3dfc0a987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.230880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e994e1-6f9a-42d8-b9a7-04d3dfc0a987 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.231152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e994e1-6f9a-42d8-b9a7-04d3dfc0a987 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.231418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e994e1-6f9a-42d8-b9a7-04d3dfc0a987 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.232591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e994e1-6f9a-42d8-b9a7-04d3dfc0a987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:28.234046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e994e1-6f9a-42d8-b9a7-04d3dfc0a987 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:28.235649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 91/188] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:28.238506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e994e1-6f9a-42d8-b9a7-04d3dfc0a987 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.287874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:28.311676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d40f3-fee7-43da-aab0-297cbad8f8cf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.348068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.348858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.349230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.349603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.349732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e994e1-6f9a-42d8-b9a7-04d3dfc0a987 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.350472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:28.350887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:28.352605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:28.366820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 98/189] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:28.367572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.385729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.386412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.386700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.386963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.387602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:28.387931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:28.389330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:28.397467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 92/190] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:28.398272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.425673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:28.476305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd88442-113d-43b1-b663-f7030682032e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.515483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b517ecbd-646c-4d73-a38e-fba7f61d9be1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.517295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.517932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.519037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.519307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.520037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] RBAC: Adding query filter params (role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=35177400b30e4b81bcbd08ec003d5756, user.id=66a9447c53c74350bc5f1a7039895a77) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:28.520338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:28.521904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:28.528747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 99/191] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/role_assignments?user.id=66a9447c53c74350bc5f1a7039895a77&scope.project.id=35177400b30e4b81bcbd08ec003d5756&role.id=7b4a508ded224dfc812c05d174d41511 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:28.529679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.541744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:28.568581 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8f584-076c-4b5f-b260-c46e9dd776af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.569389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8f584-076c-4b5f-b260-c46e9dd776af None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.569799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8f584-076c-4b5f-b260-c46e9dd776af None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.570166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8f584-076c-4b5f-b260-c46e9dd776af None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.570951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8f584-076c-4b5f-b260-c46e9dd776af None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:28.573311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8f584-076c-4b5f-b260-c46e9dd776af None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:28.607958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 93/192] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:57:28 2022] POST /identity/v3/users => generated 297 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:28.607958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8f584-076c-4b5f-b260-c46e9dd776af None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.723928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d6aa-4b45-49e4-90fc-2a1625526a89 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.763842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfbfeb-b385-4d5f-b06c-ccc7db4e34d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.764490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfbfeb-b385-4d5f-b06c-ccc7db4e34d0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.764768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfbfeb-b385-4d5f-b06c-ccc7db4e34d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.765041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccfbfeb-b385-4d5f-b06c-ccc7db4e34d0 None admin] PATH_INFO: `/v3/users/05ce0a3263774d8abdf2ad87bce296c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.781167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfbfeb-b385-4d5f-b06c-ccc7db4e34d0 None admin] RBAC: Authorizing `identity:get_user(user_id=05ce0a3263774d8abdf2ad87bce296c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:28.782777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccfbfeb-b385-4d5f-b06c-ccc7db4e34d0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:28.784876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 100/193] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/users/05ce0a3263774d8abdf2ad87bce296c1 => generated 283 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:28.792545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfbfeb-b385-4d5f-b06c-ccc7db4e34d0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.797894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:28.823970 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8f584-076c-4b5f-b260-c46e9dd776af None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.865905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8684ad5f-19a5-4809-b8cd-db67d68c3cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.866702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ad5f-19a5-4809-b8cd-db67d68c3cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.866977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ad5f-19a5-4809-b8cd-db67d68c3cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.867608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8684ad5f-19a5-4809-b8cd-db67d68c3cf5 None admin] PATH_INFO: `/v3/users/05ce0a3263774d8abdf2ad87bce296c1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.869895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684ad5f-19a5-4809-b8cd-db67d68c3cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=05ce0a3263774d8abdf2ad87bce296c1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:28.871450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8684ad5f-19a5-4809-b8cd-db67d68c3cf5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:28.874083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 94/194] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/users/05ce0a3263774d8abdf2ad87bce296c1 => generated 283 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:28.874305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd81e8-8b8c-4bc6-b871-8df3b3ee2d17 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.874575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd81e8-8b8c-4bc6-b871-8df3b3ee2d17 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.874840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfd81e8-8b8c-4bc6-b871-8df3b3ee2d17 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:28.875481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:28.878457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 95/195] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:28.882176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd81e8-8b8c-4bc6-b871-8df3b3ee2d17 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:28.921920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccfbfeb-b385-4d5f-b06c-ccc7db4e34d0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:28.966505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef469a87-6022-45f5-ab6c-673188cc9b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:28.967129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef469a87-6022-45f5-ab6c-673188cc9b1f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:28.967401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef469a87-6022-45f5-ab6c-673188cc9b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:28.967663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef469a87-6022-45f5-ab6c-673188cc9b1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:29.006074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfd81e8-8b8c-4bc6-b871-8df3b3ee2d17 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:29.045997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6012e0-9a3f-45cd-89cf-6275630b7ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:29.046532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6012e0-9a3f-45cd-89cf-6275630b7ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:29.046803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6012e0-9a3f-45cd-89cf-6275630b7ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:29.047067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6012e0-9a3f-45cd-89cf-6275630b7ec1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:29.049833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef469a87-6022-45f5-ab6c-673188cc9b1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:29.051919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef469a87-6022-45f5-ab6c-673188cc9b1f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:29.068535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:29.123135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6012e0-9a3f-45cd-89cf-6275630b7ec1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:29.124603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6012e0-9a3f-45cd-89cf-6275630b7ec1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:29.130063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:29.187038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 101/196] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:29.187225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f202-4002-4b05-b9e0-21a3361bbcba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:29.187871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f202-4002-4b05-b9e0-21a3361bbcba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:29.187871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8f202-4002-4b05-b9e0-21a3361bbcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:29.205733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5ca8f202-4002-4b05-b9e0-21a3361bbcba None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:29.206974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5ca8f202-4002-4b05-b9e0-21a3361bbcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:29.241513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 96/197] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:29.242722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6012e0-9a3f-45cd-89cf-6275630b7ec1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:29.278783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 102/198] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:29.279634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8f202-4002-4b05-b9e0-21a3361bbcba None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:29.293954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:29.353073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6012e0-9a3f-45cd-89cf-6275630b7ec1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:29.391404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99af42-bf18-4bff-bc25-318031170e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:29.392434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99af42-bf18-4bff-bc25-318031170e58 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:29.392907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99af42-bf18-4bff-bc25-318031170e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:29.393395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d99af42-bf18-4bff-bc25-318031170e58 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:29.394904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99af42-bf18-4bff-bc25-318031170e58 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:29.396693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d99af42-bf18-4bff-bc25-318031170e58 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:29.397171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:29.399649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 97/199] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:29 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:29.405905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99af42-bf18-4bff-bc25-318031170e58 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:29.410611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8f202-4002-4b05-b9e0-21a3361bbcba None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:29.475445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888855c-5e0c-4da0-bf9b-56240e8e6c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:29.476313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888855c-5e0c-4da0-bf9b-56240e8e6c89 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:29.476433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888855c-5e0c-4da0-bf9b-56240e8e6c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:29.476617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4888855c-5e0c-4da0-bf9b-56240e8e6c89 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:29.477820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888855c-5e0c-4da0-bf9b-56240e8e6c89 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:29.479245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4888855c-5e0c-4da0-bf9b-56240e8e6c89 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:29.481278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 103/200] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:29 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:29.490907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888855c-5e0c-4da0-bf9b-56240e8e6c89 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:29.520202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d99af42-bf18-4bff-bc25-318031170e58 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:29.535837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:29.564444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:29.564996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:29.565272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:29.565539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:29.566314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=bb88575700f64da982e71dffc4bac6e9, user.id=05ce0a3263774d8abdf2ad87bce296c1) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:29.566617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:29.574349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:29.586057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 98/201] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:29 2022] GET /identity/v3/role_assignments?user.id=05ce0a3263774d8abdf2ad87bce296c1&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:29.686211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4888855c-5e0c-4da0-bf9b-56240e8e6c89 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:29.727455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:29.740376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:29.741250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:29.741250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:29.741528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:29.742282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] RBAC: Adding query filter params (role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=bb88575700f64da982e71dffc4bac6e9, user.id=05ce0a3263774d8abdf2ad87bce296c1) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:29.742586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:29.744049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:29.754468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 104/202] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:29 2022] GET /identity/v3/role_assignments?user.id=05ce0a3263774d8abdf2ad87bce296c1&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=5491e6c9891843b8979a054e900d2e34 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:29.848269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01bfaa0-ec99-4c41-a12c-c650a00b1ae9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:29.889039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21934b9a-cd8f-43bb-a858-777e42738d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:29.890199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21934b9a-cd8f-43bb-a858-777e42738d02 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:29.890762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21934b9a-cd8f-43bb-a858-777e42738d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:29.891237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21934b9a-cd8f-43bb-a858-777e42738d02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:29.909594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21934b9a-cd8f-43bb-a858-777e42738d02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-21934b9a-cd8f-43bb-a858-777e42738d02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:29.916654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:29.917799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:29.918593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:29.921507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 99/203] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:29.928100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:29.965143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:30.053891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e88719-93b4-4c65-8ec5-ee7befda2c2e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:30.089145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:30.089854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:30.090134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:30.090525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:30.091267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:30.091573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:30.092988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:30.101526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 105/204] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:30.107565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21934b9a-cd8f-43bb-a858-777e42738d02 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:30.150938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:30.215624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21934b9a-cd8f-43bb-a858-777e42738d02 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:30.250741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a091b-a280-457c-859e-11606df1d305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:30.251389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a091b-a280-457c-859e-11606df1d305 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:30.251668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a091b-a280-457c-859e-11606df1d305 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:30.251937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7a091b-a280-457c-859e-11606df1d305 None admin] PATH_INFO: `/v3/users/249d095f4c484e1ab620fb17e16c8b33` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:30.253083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a091b-a280-457c-859e-11606df1d305 None admin] RBAC: Authorizing `identity:get_user(user_id=249d095f4c484e1ab620fb17e16c8b33)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:30.254609 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7a091b-a280-457c-859e-11606df1d305 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:30.256623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 100/205] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:30 2022] GET /identity/v3/users/249d095f4c484e1ab620fb17e16c8b33 => generated 289 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:30.262976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:30.368101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:30.385645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d480d1b2-65cd-4116-aed1-8c8ae27534fd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:30.431689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b344a2b-f4af-4ff5-a961-ba81b10e3341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:30.432717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b344a2b-f4af-4ff5-a961-ba81b10e3341 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:30.433195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b344a2b-f4af-4ff5-a961-ba81b10e3341 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:30.433691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b344a2b-f4af-4ff5-a961-ba81b10e3341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:30.525000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b344a2b-f4af-4ff5-a961-ba81b10e3341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:30.526520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b344a2b-f4af-4ff5-a961-ba81b10e3341 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:30.633869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:30.649907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 106/206] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:30.655200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a091b-a280-457c-859e-11606df1d305 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:30.794936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7a091b-a280-457c-859e-11606df1d305 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:30.806009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:30.835763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c37fb-d9d9-4686-8dd1-2a488a85570f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:30.836396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c37fb-d9d9-4686-8dd1-2a488a85570f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:30.836667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c37fb-d9d9-4686-8dd1-2a488a85570f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:30.836933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c37fb-d9d9-4686-8dd1-2a488a85570f None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:30.838112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c37fb-d9d9-4686-8dd1-2a488a85570f None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:30.839565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c37fb-d9d9-4686-8dd1-2a488a85570f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:30.841507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 101/207] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:30 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:30.847543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b344a2b-f4af-4ff5-a961-ba81b10e3341 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:30.992575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b344a2b-f4af-4ff5-a961-ba81b10e3341 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:31.040982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571fedc9-4277-4689-879a-43e8f93eab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:31.041599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fedc9-4277-4689-879a-43e8f93eab10 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:31.041894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fedc9-4277-4689-879a-43e8f93eab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:31.042162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571fedc9-4277-4689-879a-43e8f93eab10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:31.042878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fedc9-4277-4689-879a-43e8f93eab10 None admin] RBAC: Adding query filter params (role.id=2e8c5a373cb5440d91307a8c8ab3186d, scope.project.id=d9df1c54939a4373af874595595358c5, user.id=249d095f4c484e1ab620fb17e16c8b33) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:31.043179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fedc9-4277-4689-879a-43e8f93eab10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:31.044629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571fedc9-4277-4689-879a-43e8f93eab10 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:31.054252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 107/208] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:30 2022] GET /identity/v3/role_assignments?user.id=249d095f4c484e1ab620fb17e16c8b33&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=2e8c5a373cb5440d91307a8c8ab3186d => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:31.359052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170ce00-8cfd-45c8-bd93-6fc11fa3e53a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:31.360030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170ce00-8cfd-45c8-bd93-6fc11fa3e53a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:31.360605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170ce00-8cfd-45c8-bd93-6fc11fa3e53a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:31.362757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 102/209] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:31.368668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcf6ed-dde4-46c9-b779-8926ef8eda63 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:31.368991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcf6ed-dde4-46c9-b779-8926ef8eda63 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:31.369210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbcf6ed-dde4-46c9-b779-8926ef8eda63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:31.397381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9dbcf6ed-dde4-46c9-b779-8926ef8eda63 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:31.398656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9dbcf6ed-dde4-46c9-b779-8926ef8eda63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:31.453567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:31.476909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 108/210] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:31.575726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d56bc7-9b21-471d-a419-89bd1eb02af5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:31.576087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d56bc7-9b21-471d-a419-89bd1eb02af5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:31.576279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d56bc7-9b21-471d-a419-89bd1eb02af5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:31.577818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 103/211] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:31.583172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e5d6f-dd0a-4feb-9924-ffa6ecd69ee3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:31.583458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e5d6f-dd0a-4feb-9924-ffa6ecd69ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:31.583729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4e5d6f-dd0a-4feb-9924-ffa6ecd69ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:31.605612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0b4e5d6f-dd0a-4feb-9924-ffa6ecd69ee3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:31.606818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0b4e5d6f-dd0a-4feb-9924-ffa6ecd69ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:31.679279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 109/212] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:31.772245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d56bc7-9b21-471d-a419-89bd1eb02af5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:31.893790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d56bc7-9b21-471d-a419-89bd1eb02af5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:31.941797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29e345-1368-4df2-b2c9-93433386acd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:31.942817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29e345-1368-4df2-b2c9-93433386acd7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:31.943290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29e345-1368-4df2-b2c9-93433386acd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:31.943763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29e345-1368-4df2-b2c9-93433386acd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:31.955229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba29e345-1368-4df2-b2c9-93433386acd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:31.957842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e5d6f-dd0a-4feb-9924-ffa6ecd69ee3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ba29e345-1368-4df2-b2c9-93433386acd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:31.960648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:31.961529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:31.962349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:31.965710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 104/213] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:31.969150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29e345-1368-4df2-b2c9-93433386acd7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:32.015661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:32.072835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4e5d6f-dd0a-4feb-9924-ffa6ecd69ee3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:32.085476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:32.109177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba29e345-1368-4df2-b2c9-93433386acd7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:32.122078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e51e57-f9f9-4ad2-8751-c6dc4f5b637e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:32.123145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e51e57-f9f9-4ad2-8751-c6dc4f5b637e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.124088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e51e57-f9f9-4ad2-8751-c6dc4f5b637e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.124546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e51e57-f9f9-4ad2-8751-c6dc4f5b637e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.126640 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e51e57-f9f9-4ad2-8751-c6dc4f5b637e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:32.129010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e51e57-f9f9-4ad2-8751-c6dc4f5b637e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:32.131460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 110/214] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:32.137556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e51e57-f9f9-4ad2-8751-c6dc4f5b637e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:32.154987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:32.155613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.155888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.156157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.156821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:32.157122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:32.158646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:32.172413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 105/215] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:32.173675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:32.257055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e51e57-f9f9-4ad2-8751-c6dc4f5b637e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:32.283928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:32.290200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b41e92-ee17-4cdd-9714-01bae72d92eb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:32.297021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765645d-5c30-48f4-8ebc-43396df91c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:32.297671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765645d-5c30-48f4-8ebc-43396df91c0b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.297945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765645d-5c30-48f4-8ebc-43396df91c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.298207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765645d-5c30-48f4-8ebc-43396df91c0b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.298857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765645d-5c30-48f4-8ebc-43396df91c0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:32.300300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5765645d-5c30-48f4-8ebc-43396df91c0b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:32.325317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44af070-ac9e-4251-9a60-f8c3c2a8789d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:32.326011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44af070-ac9e-4251-9a60-f8c3c2a8789d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.326283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44af070-ac9e-4251-9a60-f8c3c2a8789d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.326550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44af070-ac9e-4251-9a60-f8c3c2a8789d None admin] PATH_INFO: `/v3/users/b0f78890553e43e8ac446cbdab22c935` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.329698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 111/216] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:57:32 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:32.329955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82834f58-ea65-44c6-b584-c7c791767324 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.330142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82834f58-ea65-44c6-b584-c7c791767324 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.330410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82834f58-ea65-44c6-b584-c7c791767324 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.336275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 112/217] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:32.336275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fbeca-a128-449f-95f6-70e5371ff4ef None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.336275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fbeca-a128-449f-95f6-70e5371ff4ef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.336275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064fbeca-a128-449f-95f6-70e5371ff4ef None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.336275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 113/218] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:32 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:32.339613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddb4bf-76bc-431a-bfd2-61fda0c6be01 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.339899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddb4bf-76bc-431a-bfd2-61fda0c6be01 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.340146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfddb4bf-76bc-431a-bfd2-61fda0c6be01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.344649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44af070-ac9e-4251-9a60-f8c3c2a8789d None admin] RBAC: Authorizing `identity:get_user(user_id=b0f78890553e43e8ac446cbdab22c935)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:32.346394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44af070-ac9e-4251-9a60-f8c3c2a8789d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:32.349690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 106/219] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:32 2022] GET /identity/v3/users/b0f78890553e43e8ac446cbdab22c935 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:32.352225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709da82-019f-4b13-a60e-a5bcd8e71925 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.352935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709da82-019f-4b13-a60e-a5bcd8e71925 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.353564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709da82-019f-4b13-a60e-a5bcd8e71925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.361423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bfddb4bf-76bc-431a-bfd2-61fda0c6be01 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:32.362996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bfddb4bf-76bc-431a-bfd2-61fda0c6be01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:32.373460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d709da82-019f-4b13-a60e-a5bcd8e71925 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:32.374757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d709da82-019f-4b13-a60e-a5bcd8e71925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:32.401297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:32.448737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 114/220] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:32.457661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 107/221] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:32.457661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709da82-019f-4b13-a60e-a5bcd8e71925 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:32.548084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:32.564826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709da82-019f-4b13-a60e-a5bcd8e71925 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:32.598757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc84a4d-3958-4ee0-9487-3197d5434a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:32.599393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc84a4d-3958-4ee0-9487-3197d5434a75 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.599666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc84a4d-3958-4ee0-9487-3197d5434a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.599928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc84a4d-3958-4ee0-9487-3197d5434a75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.673140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc84a4d-3958-4ee0-9487-3197d5434a75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:32.674626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc84a4d-3958-4ee0-9487-3197d5434a75 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:32.709709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddb4bf-76bc-431a-bfd2-61fda0c6be01 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:32.776434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:32.798104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 108/222] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:32.798987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc84a4d-3958-4ee0-9487-3197d5434a75 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:32.837794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfddb4bf-76bc-431a-bfd2-61fda0c6be01 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:32.884920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eda01f-4f73-405d-9d7c-05acb6bbf6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:32.886052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eda01f-4f73-405d-9d7c-05acb6bbf6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.886338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eda01f-4f73-405d-9d7c-05acb6bbf6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.886609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eda01f-4f73-405d-9d7c-05acb6bbf6d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.901034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eda01f-4f73-405d-9d7c-05acb6bbf6d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-49eda01f-4f73-405d-9d7c-05acb6bbf6d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:32.906088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:32.907055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:32.907842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 115/223] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:32.908471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eda01f-4f73-405d-9d7c-05acb6bbf6d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:32.914093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc84a4d-3958-4ee0-9487-3197d5434a75 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:32.935901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:32.951773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d52c6e-4b29-4e23-83d7-25f309958e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:32.952536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d52c6e-4b29-4e23-83d7-25f309958e55 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:32.952872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d52c6e-4b29-4e23-83d7-25f309958e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:32.953203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d52c6e-4b29-4e23-83d7-25f309958e55 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:32.964941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d52c6e-4b29-4e23-83d7-25f309958e55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e3d52c6e-4b29-4e23-83d7-25f309958e55 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:32.969891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:32.970960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:32.971909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:32.972562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 109/224] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:32.972872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d52c6e-4b29-4e23-83d7-25f309958e55 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.027828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:33.042850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eda01f-4f73-405d-9d7c-05acb6bbf6d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.086770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d52c6e-4b29-4e23-83d7-25f309958e55 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.091071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203580b-7ec4-448f-ad4a-cbc27605411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.092268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203580b-7ec4-448f-ad4a-cbc27605411c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.092690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203580b-7ec4-448f-ad4a-cbc27605411c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.093101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2203580b-7ec4-448f-ad4a-cbc27605411c None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.094479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203580b-7ec4-448f-ad4a-cbc27605411c None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.096651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2203580b-7ec4-448f-ad4a-cbc27605411c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.099713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 116/225] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:32 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:33.100945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203580b-7ec4-448f-ad4a-cbc27605411c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.125748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.126342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.126615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.126883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.127550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:33.127847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.129288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.139387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 110/226] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:33.140302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.206949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:33.216814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2203580b-7ec4-448f-ad4a-cbc27605411c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.251210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.251825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.252102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.252598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.253031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:33.253331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.254947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.263634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 117/227] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:33.268772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.274521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f78fc7-de0d-4e86-baa1-e572fcc425ae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.283525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:33.325597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.326572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.327601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.328020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.329223 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] RBAC: Adding query filter params (role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=bb88575700f64da982e71dffc4bac6e9, user.id=b0f78890553e43e8ac446cbdab22c935) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:33.329946 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.332301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.340622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 111/228] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:33 2022] GET /identity/v3/role_assignments?user.id=b0f78890553e43e8ac446cbdab22c935&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=7b4a508ded224dfc812c05d174d41511 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:33.342586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.379355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c287c1ce-e92b-4647-9c1e-48d62678ac4e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.420906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def8e0-4c25-4135-b9dd-115c06366e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.421870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def8e0-4c25-4135-b9dd-115c06366e34 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.422290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def8e0-4c25-4135-b9dd-115c06366e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.422719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65def8e0-4c25-4135-b9dd-115c06366e34 None admin] PATH_INFO: `/v3/users/05ce0a3263774d8abdf2ad87bce296c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.424862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def8e0-4c25-4135-b9dd-115c06366e34 None admin] RBAC: Authorizing `identity:get_user(user_id=05ce0a3263774d8abdf2ad87bce296c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.426528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65def8e0-4c25-4135-b9dd-115c06366e34 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.428808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 118/229] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:33 2022] GET /identity/v3/users/05ce0a3263774d8abdf2ad87bce296c1 => generated 283 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:33.435047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def8e0-4c25-4135-b9dd-115c06366e34 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.484211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:33.542225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce97139-8696-4c26-a2c8-430bf141ec58 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.595352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bc6f3-132c-4ce8-82be-dd2ab0422e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.596070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bc6f3-132c-4ce8-82be-dd2ab0422e22 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.596453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bc6f3-132c-4ce8-82be-dd2ab0422e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.596727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bc6f3-132c-4ce8-82be-dd2ab0422e22 None admin] PATH_INFO: `/v3/users/05ce0a3263774d8abdf2ad87bce296c1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.597897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bc6f3-132c-4ce8-82be-dd2ab0422e22 None admin] RBAC: Authorizing `identity:get_user(user_id=05ce0a3263774d8abdf2ad87bce296c1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.599466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bc6f3-132c-4ce8-82be-dd2ab0422e22 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.601100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 112/230] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:33 2022] GET /identity/v3/users/05ce0a3263774d8abdf2ad87bce296c1 => generated 283 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:33.606639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bc6f3-132c-4ce8-82be-dd2ab0422e22 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.650844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:33.675764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65def8e0-4c25-4135-b9dd-115c06366e34 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.715462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bc6f3-132c-4ce8-82be-dd2ab0422e22 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.751032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0465d80-b88f-4e93-8b8b-8f02132d221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.751951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465d80-b88f-4e93-8b8b-8f02132d221c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.752232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465d80-b88f-4e93-8b8b-8f02132d221c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.752502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0465d80-b88f-4e93-8b8b-8f02132d221c None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.753651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0465d80-b88f-4e93-8b8b-8f02132d221c None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.755326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0465d80-b88f-4e93-8b8b-8f02132d221c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.757028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 113/231] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:33 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:33.762699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0465d80-b88f-4e93-8b8b-8f02132d221c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.770991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798464fa-3a88-40bc-942e-241e0dba0451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.771656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798464fa-3a88-40bc-942e-241e0dba0451 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.772096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798464fa-3a88-40bc-942e-241e0dba0451 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.772360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798464fa-3a88-40bc-942e-241e0dba0451 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.778332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798464fa-3a88-40bc-942e-241e0dba0451 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.779803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798464fa-3a88-40bc-942e-241e0dba0451 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.781767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 119/232] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:33 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:33.787212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798464fa-3a88-40bc-942e-241e0dba0451 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:33.869696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0465d80-b88f-4e93-8b8b-8f02132d221c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:33.877266 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:33.917844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da49c11-4e65-4af0-b383-2e9149034af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:33.918545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da49c11-4e65-4af0-b383-2e9149034af7 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:33.918818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da49c11-4e65-4af0-b383-2e9149034af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:33.919083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da49c11-4e65-4af0-b383-2e9149034af7 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9/users/05ce0a3263774d8abdf2ad87bce296c1/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:33.921209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da49c11-4e65-4af0-b383-2e9149034af7 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb88575700f64da982e71dffc4bac6e9, user_id=05ce0a3263774d8abdf2ad87bce296c1, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:33.924894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da49c11-4e65-4af0-b383-2e9149034af7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:33.951219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 114/233] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:33 2022] PUT /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9/users/05ce0a3263774d8abdf2ad87bce296c1/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 13:57:33.955222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:33.980757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798464fa-3a88-40bc-942e-241e0dba0451 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:34.030462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ae88bc-59cc-4208-9021-5c91cd0c492d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:34.031149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae88bc-59cc-4208-9021-5c91cd0c492d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:34.031422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae88bc-59cc-4208-9021-5c91cd0c492d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:34.031691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ae88bc-59cc-4208-9021-5c91cd0c492d None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9/users/05ce0a3263774d8abdf2ad87bce296c1/roles/5491e6c9891843b8979a054e900d2e34` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:34.033943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae88bc-59cc-4208-9021-5c91cd0c492d None admin] RBAC: Authorizing `identity:create_grant(project_id=bb88575700f64da982e71dffc4bac6e9, user_id=05ce0a3263774d8abdf2ad87bce296c1, role_id=5491e6c9891843b8979a054e900d2e34)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:34.035462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ae88bc-59cc-4208-9021-5c91cd0c492d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:34.058514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 120/234] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:33 2022] PUT /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9/users/05ce0a3263774d8abdf2ad87bce296c1/roles/5491e6c9891843b8979a054e900d2e34 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 13:57:34.883112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf7d4-11d0-4dc6-bfc5-75cec0bcc333 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:34.883608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf7d4-11d0-4dc6-bfc5-75cec0bcc333 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:34.883608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbf7d4-11d0-4dc6-bfc5-75cec0bcc333 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:34.885056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 115/235] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:34.890646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0cd99-5c0e-4a71-9c85-55020aa5d40e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:34.891462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0cd99-5c0e-4a71-9c85-55020aa5d40e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:34.891978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0cd99-5c0e-4a71-9c85-55020aa5d40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:34.911132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-42d0cd99-5c0e-4a71-9c85-55020aa5d40e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:34.912670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-42d0cd99-5c0e-4a71-9c85-55020aa5d40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:34.982107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 121/236] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:35.267387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f456b-ee6f-41e9-a2c6-35d99896a086 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:35.268155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f456b-ee6f-41e9-a2c6-35d99896a086 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:35.268643 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435f456b-ee6f-41e9-a2c6-35d99896a086 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:35.270235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 116/237] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:35.281361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e053409-efb1-478d-9e70-f35816bdc8c2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:35.281659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e053409-efb1-478d-9e70-f35816bdc8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:35.281926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e053409-efb1-478d-9e70-f35816bdc8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:35.306586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6e053409-efb1-478d-9e70-f35816bdc8c2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:35.307770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6e053409-efb1-478d-9e70-f35816bdc8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:35.358368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f456b-ee6f-41e9-a2c6-35d99896a086 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:35.389709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 122/238] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:35.516744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435f456b-ee6f-41e9-a2c6-35d99896a086 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:35.560405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8b809-ed78-473b-b23f-476bfa34f73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:35.560952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b809-ed78-473b-b23f-476bfa34f73d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:35.561229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b809-ed78-473b-b23f-476bfa34f73d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:35.561497 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8b809-ed78-473b-b23f-476bfa34f73d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:35.562683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8b809-ed78-473b-b23f-476bfa34f73d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:35.564144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8b809-ed78-473b-b23f-476bfa34f73d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:35.566027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 117/239] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:35.571544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e053409-efb1-478d-9e70-f35816bdc8c2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:35.590439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:35.699335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e053409-efb1-478d-9e70-f35816bdc8c2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:35.717232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8b809-ed78-473b-b23f-476bfa34f73d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:35.740530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564091f-a9e5-4c16-8691-8cd05e3a5251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:35.741532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564091f-a9e5-4c16-8691-8cd05e3a5251 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:35.742074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564091f-a9e5-4c16-8691-8cd05e3a5251 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:35.742537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564091f-a9e5-4c16-8691-8cd05e3a5251 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:35.743462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564091f-a9e5-4c16-8691-8cd05e3a5251 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:35.745112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564091f-a9e5-4c16-8691-8cd05e3a5251 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:35.776780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 123/240] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:57:35 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:35.811800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:35.921806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8b809-ed78-473b-b23f-476bfa34f73d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:35.973281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada282a-35d2-435d-b1e4-225e64e7d4d2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:35.973852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada282a-35d2-435d-b1e4-225e64e7d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:35.973926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada282a-35d2-435d-b1e4-225e64e7d4d2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:35.975322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 124/241] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:57:35.976078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa99ef-f368-4e75-aaaf-1f435469fd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:35.976730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa99ef-f368-4e75-aaaf-1f435469fd2d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:35.977000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa99ef-f368-4e75-aaaf-1f435469fd2d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:35.977264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaa99ef-f368-4e75-aaaf-1f435469fd2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:35.984997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563b23-587f-4ddf-89de-d6091f93e97e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:35.985312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563b23-587f-4ddf-89de-d6091f93e97e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:35.985562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab563b23-587f-4ddf-89de-d6091f93e97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:36.008185 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaa99ef-f368-4e75-aaaf-1f435469fd2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7eaa99ef-f368-4e75-aaaf-1f435469fd2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:36.013966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:36.014979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:36.015942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:36.018016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 118/242] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:35 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:36.023655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ab563b23-587f-4ddf-89de-d6091f93e97e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:36.024293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa99ef-f368-4e75-aaaf-1f435469fd2d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:36.025005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab563b23-587f-4ddf-89de-d6091f93e97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:36.120937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:36.137654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 125/243] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:36.151541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:36.158257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaa99ef-f368-4e75-aaaf-1f435469fd2d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:36.192255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:36.192881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:36.193154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:36.193419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:36.194107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:36.194408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:36.195777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:36.204365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 119/244] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:36.210944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab563b23-587f-4ddf-89de-d6091f93e97e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:36.326017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab563b23-587f-4ddf-89de-d6091f93e97e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:36.365934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14ce1-14f1-4017-bdd4-b50944fb99df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:36.367116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14ce1-14f1-4017-bdd4-b50944fb99df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:36.367589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14ce1-14f1-4017-bdd4-b50944fb99df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:36.368061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14ce1-14f1-4017-bdd4-b50944fb99df None admin] PATH_INFO: `/v3/users/1adcfd23be71456996bca91341df02a0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:36.395628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c14ce1-14f1-4017-bdd4-b50944fb99df None admin] RBAC: Authorizing `identity:get_user(user_id=1adcfd23be71456996bca91341df02a0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:36.395628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c14ce1-14f1-4017-bdd4-b50944fb99df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:36.396084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 126/245] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:36 2022] GET /identity/v3/users/1adcfd23be71456996bca91341df02a0 => generated 297 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:36.406824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:36.442608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14ce1-14f1-4017-bdd4-b50944fb99df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:36.526653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:36.541508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b27fd-31f8-4f5b-9b76-af4886e6accb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:36.586752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14ce1-14f1-4017-bdd4-b50944fb99df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:36.596705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38b2a91-86d1-4632-a40f-25ee74123a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:36.597791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2a91-86d1-4632-a40f-25ee74123a81 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:36.598309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2a91-86d1-4632-a40f-25ee74123a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:36.598803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38b2a91-86d1-4632-a40f-25ee74123a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:36.629956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:36.637217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701410de-fd5d-40f0-b576-3689ccd047a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:36.637897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701410de-fd5d-40f0-b576-3689ccd047a8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:36.638170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701410de-fd5d-40f0-b576-3689ccd047a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:36.638484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701410de-fd5d-40f0-b576-3689ccd047a8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:36.649530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701410de-fd5d-40f0-b576-3689ccd047a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-701410de-fd5d-40f0-b576-3689ccd047a8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:36.654307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:36.655320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:36.656189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 127/246] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:36.658209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeffac-d563-4c01-8ad4-d380da100e1b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:36.658487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeffac-d563-4c01-8ad4-d380da100e1b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:36.658958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfeffac-d563-4c01-8ad4-d380da100e1b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:36.660218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 128/247] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:36.665303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeffac-d563-4c01-8ad4-d380da100e1b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:36.683785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b2a91-86d1-4632-a40f-25ee74123a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:36.685706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38b2a91-86d1-4632-a40f-25ee74123a81 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:36.859579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfeffac-d563-4c01-8ad4-d380da100e1b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:36.882776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 120/248] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:36.886006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8ada4-c00b-4a11-9d33-6ac5268cf030 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:36.886299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8ada4-c00b-4a11-9d33-6ac5268cf030 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:36.886546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd8ada4-c00b-4a11-9d33-6ac5268cf030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:36.894362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:36.922202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fcd8ada4-c00b-4a11-9d33-6ac5268cf030 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:36.923394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fcd8ada4-c00b-4a11-9d33-6ac5268cf030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:36.936713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:36.937338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:36.937611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:36.937926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:36.938593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:36.938889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:36.940277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:36.955118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 129/249] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:36.955978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:36.998257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:37.016459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 121/250] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:37.017604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8ada4-c00b-4a11-9d33-6ac5268cf030 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:37.087380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61788c7-6491-427b-ac23-30b6a5a02c23 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:37.121981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f882f-5a89-4da2-b201-c8358a633855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:37.122953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f882f-5a89-4da2-b201-c8358a633855 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.123400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f882f-5a89-4da2-b201-c8358a633855 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.123837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f882f-5a89-4da2-b201-c8358a633855 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.125434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f882f-5a89-4da2-b201-c8358a633855 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:37.127096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f882f-5a89-4da2-b201-c8358a633855 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:37.128938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 130/251] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:36 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:37.137399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019148d9-7748-452e-80c0-530506ce2179 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.138196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019148d9-7748-452e-80c0-530506ce2179 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.138666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019148d9-7748-452e-80c0-530506ce2179 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.140765 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 131/252] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:37 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:37.141741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019148d9-7748-452e-80c0-530506ce2179 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:37.148365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd8ada4-c00b-4a11-9d33-6ac5268cf030 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:37.185774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7555845-a5e4-4300-afa0-bd5b24cc5728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:37.187017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7555845-a5e4-4300-afa0-bd5b24cc5728 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.187308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7555845-a5e4-4300-afa0-bd5b24cc5728 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.188182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7555845-a5e4-4300-afa0-bd5b24cc5728 None admin] PATH_INFO: `/v3/users/b0f78890553e43e8ac446cbdab22c935` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.190194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7555845-a5e4-4300-afa0-bd5b24cc5728 None admin] RBAC: Authorizing `identity:get_user(user_id=b0f78890553e43e8ac446cbdab22c935)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:37.193457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7555845-a5e4-4300-afa0-bd5b24cc5728 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:37.197645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 122/253] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:37 2022] GET /identity/v3/users/b0f78890553e43e8ac446cbdab22c935 => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:37.200789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae2421-4934-430e-b17f-36bd6e6667a8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.201914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae2421-4934-430e-b17f-36bd6e6667a8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.202157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bae2421-4934-430e-b17f-36bd6e6667a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.231850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:37.239985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1bae2421-4934-430e-b17f-36bd6e6667a8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:37.241236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1bae2421-4934-430e-b17f-36bd6e6667a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:37.325441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:37.333602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019148d9-7748-452e-80c0-530506ce2179 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:37.365430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 123/254] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:37.366568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae2421-4934-430e-b17f-36bd6e6667a8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:37.371969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:37.372597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.372870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.373179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.373985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] RBAC: Adding query filter params (role.id=2e8c5a373cb5440d91307a8c8ab3186d, scope.project.id=bb88575700f64da982e71dffc4bac6e9, user.id=1adcfd23be71456996bca91341df02a0) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:37.374287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:37.375881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:37.386883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 132/255] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:37 2022] GET /identity/v3/role_assignments?user.id=1adcfd23be71456996bca91341df02a0&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=2e8c5a373cb5440d91307a8c8ab3186d => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:37.405498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:37.522018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bae2421-4934-430e-b17f-36bd6e6667a8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:37.539023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16234b2-7ed3-4cff-b244-c8d1b7bae4ea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:37.553271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:37.587062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e15c7d-258a-429f-814f-63aaa430e402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:37.587725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e15c7d-258a-429f-814f-63aaa430e402 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.587992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e15c7d-258a-429f-814f-63aaa430e402 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.588255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e15c7d-258a-429f-814f-63aaa430e402 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.590433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e15c7d-258a-429f-814f-63aaa430e402 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:37.591880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e15c7d-258a-429f-814f-63aaa430e402 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:37.594638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 124/256] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:37 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:37.600597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e15c7d-258a-429f-814f-63aaa430e402 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:37.606448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ccdb8-ee5e-4a69-84e0-43e3a2a04be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:37.607106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccdb8-ee5e-4a69-84e0-43e3a2a04be4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.612199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccdb8-ee5e-4a69-84e0-43e3a2a04be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.612512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ccdb8-ee5e-4a69-84e0-43e3a2a04be4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.631358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ccdb8-ee5e-4a69-84e0-43e3a2a04be4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-622ccdb8-ee5e-4a69-84e0-43e3a2a04be4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:37.636675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:37.637546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:37.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:37.640555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 133/257] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:37.650084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ccdb8-ee5e-4a69-84e0-43e3a2a04be4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:37.709342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:37.806853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e15c7d-258a-429f-814f-63aaa430e402 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:37.829105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ccdb8-ee5e-4a69-84e0-43e3a2a04be4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:37.865156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1724e85-4ffb-4de4-8ab1-4c4bbdb7fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:37.866358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1724e85-4ffb-4de4-8ab1-4c4bbdb7fb39 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.867767 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1724e85-4ffb-4de4-8ab1-4c4bbdb7fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.868176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1724e85-4ffb-4de4-8ab1-4c4bbdb7fb39 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9/users/b0f78890553e43e8ac446cbdab22c935/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.872197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1724e85-4ffb-4de4-8ab1-4c4bbdb7fb39 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb88575700f64da982e71dffc4bac6e9, user_id=b0f78890553e43e8ac446cbdab22c935, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:37.874960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1724e85-4ffb-4de4-8ab1-4c4bbdb7fb39 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:37.888292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:37.890625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:37.892502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:37.893171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:37.894552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:37.897232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:37.898796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:37.898995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 125/258] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:37 2022] PUT /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9/users/b0f78890553e43e8ac446cbdab22c935/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:37.900092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1724e85-4ffb-4de4-8ab1-4c4bbdb7fb39 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:37.921083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 134/259] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:37.926647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:37.994381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:38.109753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1724e85-4ffb-4de4-8ab1-4c4bbdb7fb39 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:38.131777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04d3cd-a9d0-421b-b883-22fa315460cd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:38.151206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:38.158961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a99fb-e26d-42e3-8f12-c3c390e20614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:38.159652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a99fb-e26d-42e3-8f12-c3c390e20614 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:38.159931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a99fb-e26d-42e3-8f12-c3c390e20614 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:38.160211 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a99fb-e26d-42e3-8f12-c3c390e20614 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:38.189117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41337ae8-3593-4a25-93d9-d1bc7603ae77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:38.189792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a99fb-e26d-42e3-8f12-c3c390e20614 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:38.189903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337ae8-3593-4a25-93d9-d1bc7603ae77 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:38.190102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337ae8-3593-4a25-93d9-d1bc7603ae77 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:38.190366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41337ae8-3593-4a25-93d9-d1bc7603ae77 None admin] PATH_INFO: `/v3/users/05ce0a3263774d8abdf2ad87bce296c1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:38.191576 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41337ae8-3593-4a25-93d9-d1bc7603ae77 None admin] RBAC: Authorizing `identity:get_user(user_id=05ce0a3263774d8abdf2ad87bce296c1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:38.193074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41337ae8-3593-4a25-93d9-d1bc7603ae77 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-208a99fb-e26d-42e3-8f12-c3c390e20614 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:38.195052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:38.196017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:38.196798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:38.198969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 135/260] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:37 2022] GET /identity/v3/users/05ce0a3263774d8abdf2ad87bce296c1 => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:38.199052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 126/261] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:38.201879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41337ae8-3593-4a25-93d9-d1bc7603ae77 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:38.207069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a99fb-e26d-42e3-8f12-c3c390e20614 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:38.388237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:38.394741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41337ae8-3593-4a25-93d9-d1bc7603ae77 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:38.408203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a99fb-e26d-42e3-8f12-c3c390e20614 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:38.461795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72182a3-384a-4057-8013-c13a99941615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:38.462444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72182a3-384a-4057-8013-c13a99941615 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:38.462714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72182a3-384a-4057-8013-c13a99941615 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:38.462977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72182a3-384a-4057-8013-c13a99941615 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:38.477659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:38.478273 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:38.478544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:38.478808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:38.479474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:38.479769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:38.481159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:38.496591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 127/262] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:38.504132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:38.589000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:38.642321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72182a3-384a-4057-8013-c13a99941615 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:38.645045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72182a3-384a-4057-8013-c13a99941615 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:38.740831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f8e7-4bd9-433a-884f-279858a1a477 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:38.801790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9f287-1970-4e3e-a61a-2f16dcde66f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:38.802553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f287-1970-4e3e-a61a-2f16dcde66f1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:38.802833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f287-1970-4e3e-a61a-2f16dcde66f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:38.803098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a9f287-1970-4e3e-a61a-2f16dcde66f1 None admin] PATH_INFO: `/v3/users/05ce0a3263774d8abdf2ad87bce296c1` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:38.807045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9f287-1970-4e3e-a61a-2f16dcde66f1 None admin] RBAC: Authorizing `identity:get_user(user_id=05ce0a3263774d8abdf2ad87bce296c1)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:38.808511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a9f287-1970-4e3e-a61a-2f16dcde66f1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:38.811455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 128/263] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:38 2022] GET /identity/v3/users/05ce0a3263774d8abdf2ad87bce296c1 => generated 283 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:38.817664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9f287-1970-4e3e-a61a-2f16dcde66f1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:38.873392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:38.884130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 136/264] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:38.885293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72182a3-384a-4057-8013-c13a99941615 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:38.999439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a9f287-1970-4e3e-a61a-2f16dcde66f1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:39.000979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:39.070911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc92ec6-d110-4a07-8910-722e6d7fcea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:39.071534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc92ec6-d110-4a07-8910-722e6d7fcea9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.071805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc92ec6-d110-4a07-8910-722e6d7fcea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.072069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc92ec6-d110-4a07-8910-722e6d7fcea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.098718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72182a3-384a-4057-8013-c13a99941615 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:39.159809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1ec0e8-4f87-440b-8aa1-7843f156a45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:39.160604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ec0e8-4f87-440b-8aa1-7843f156a45e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.160828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ec0e8-4f87-440b-8aa1-7843f156a45e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.161165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1ec0e8-4f87-440b-8aa1-7843f156a45e None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.162351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ec0e8-4f87-440b-8aa1-7843f156a45e None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:39.163804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1ec0e8-4f87-440b-8aa1-7843f156a45e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:39.165797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 137/265] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:38 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:39.167184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2e3b-c6ba-4e5f-881c-7f5b36bd8f4e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.167460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2e3b-c6ba-4e5f-881c-7f5b36bd8f4e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.167724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c2e3b-c6ba-4e5f-881c-7f5b36bd8f4e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.168921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 138/266] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:39.173868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c2e3b-c6ba-4e5f-881c-7f5b36bd8f4e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:39.222229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc92ec6-d110-4a07-8910-722e6d7fcea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:39.223825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc92ec6-d110-4a07-8910-722e6d7fcea9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:39.274071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:39.350509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c2e3b-c6ba-4e5f-881c-7f5b36bd8f4e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:39.362147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 129/267] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:39.363846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fffd5a-0206-4c49-96b1-fe3ccd0db0c2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.364390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fffd5a-0206-4c49-96b1-fe3ccd0db0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.364864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fffd5a-0206-4c49-96b1-fe3ccd0db0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.376653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:39.402541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b2fffd5a-0206-4c49-96b1-fe3ccd0db0c2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:39.404805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b2fffd5a-0206-4c49-96b1-fe3ccd0db0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:39.454366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:39.454994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.455267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.455531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.456267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=bb88575700f64da982e71dffc4bac6e9, user.id=05ce0a3263774d8abdf2ad87bce296c1) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:39.456575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:39.458173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:39.464582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 139/268] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:39 2022] GET /identity/v3/role_assignments?user.id=05ce0a3263774d8abdf2ad87bce296c1&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 608 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:39.465204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:39.486020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 130/269] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:39.488101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477289f1-4cdb-404f-927b-0fb631ef7fa4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.488582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477289f1-4cdb-404f-927b-0fb631ef7fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.489111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477289f1-4cdb-404f-927b-0fb631ef7fa4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.490778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 131/270] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:39.497447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22448167-6705-40cc-b246-5c4afa3e68e7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.498116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22448167-6705-40cc-b246-5c4afa3e68e7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.498590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22448167-6705-40cc-b246-5c4afa3e68e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.533974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-22448167-6705-40cc-b246-5c4afa3e68e7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:39.535742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-22448167-6705-40cc-b246-5c4afa3e68e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:39.643532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091429-eb67-47fc-a925-af5de6fbc465 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:39.690055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:39.703876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce6f6c-9879-4c51-8f7a-dc7c843a7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:39.704538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce6f6c-9879-4c51-8f7a-dc7c843a7172 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.704817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce6f6c-9879-4c51-8f7a-dc7c843a7172 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.705077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ce6f6c-9879-4c51-8f7a-dc7c843a7172 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.710642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce6f6c-9879-4c51-8f7a-dc7c843a7172 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:39.712159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ce6f6c-9879-4c51-8f7a-dc7c843a7172 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:39.714266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 140/271] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:39 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:39.720245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce6f6c-9879-4c51-8f7a-dc7c843a7172 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:39.735634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 132/272] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:39.845831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22448167-6705-40cc-b246-5c4afa3e68e7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:39.875500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ce6f6c-9879-4c51-8f7a-dc7c843a7172 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:39.919152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:39.919794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:39.920052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:39.921577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:39.922342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] RBAC: Adding query filter params (role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=bb88575700f64da982e71dffc4bac6e9, user.id=05ce0a3263774d8abdf2ad87bce296c1) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:39.922665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:39.924101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:39.936982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 141/273] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:39 2022] GET /identity/v3/role_assignments?user.id=05ce0a3263774d8abdf2ad87bce296c1&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=5491e6c9891843b8979a054e900d2e34 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:39.958089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:40.000793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22448167-6705-40cc-b246-5c4afa3e68e7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:40.042588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:40.071552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c70b-1ca5-4dd8-8704-9650d2f3c75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:40.072461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c70b-1ca5-4dd8-8704-9650d2f3c75c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.072900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c70b-1ca5-4dd8-8704-9650d2f3c75c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.073396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9c70b-1ca5-4dd8-8704-9650d2f3c75c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.096214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9c70b-1ca5-4dd8-8704-9650d2f3c75c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3be9c70b-1ca5-4dd8-8704-9650d2f3c75c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:40.102132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:40.103193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:40.104117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:40.109810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 133/274] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:40.112645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c70b-1ca5-4dd8-8704-9650d2f3c75c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:40.139220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:40.194680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f355bad-76ee-48f7-bd70-6044c1707637 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:40.237294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06020fec-190f-438b-92c8-bfa0a99f4eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:40.238241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06020fec-190f-438b-92c8-bfa0a99f4eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.238526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06020fec-190f-438b-92c8-bfa0a99f4eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.238792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06020fec-190f-438b-92c8-bfa0a99f4eb9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.240438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06020fec-190f-438b-92c8-bfa0a99f4eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:40.241933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06020fec-190f-438b-92c8-bfa0a99f4eb9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:40.244365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 142/275] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:40.249558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06020fec-190f-438b-92c8-bfa0a99f4eb9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:40.263188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9c70b-1ca5-4dd8-8704-9650d2f3c75c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:40.292421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:40.311890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:40.312537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.312810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.313075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.313782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:40.314145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:40.315521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:40.326212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 134/276] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:40.332349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:40.382847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06020fec-190f-438b-92c8-bfa0a99f4eb9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:40.413454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:40.421370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adadc615-97b4-4507-b590-7d7d21c862dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:40.422373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc615-97b4-4507-b590-7d7d21c862dc None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.422851 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc615-97b4-4507-b590-7d7d21c862dc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.423343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adadc615-97b4-4507-b590-7d7d21c862dc None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.424217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadc615-97b4-4507-b590-7d7d21c862dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:40.425973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adadc615-97b4-4507-b590-7d7d21c862dc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:40.455707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015279e-8d95-4d97-8662-c776892dc8f6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:40.461377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 143/277] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:57:40 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:40.470630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f4cc3-be7d-427a-bca9-c4063ccb6c2c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.470959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f4cc3-be7d-427a-bca9-c4063ccb6c2c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.472079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77f4cc3-be7d-427a-bca9-c4063ccb6c2c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.472908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 144/278] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:57:40.479219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5731-a44b-4aa3-8b8b-be86f92cf7ac None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.479557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5731-a44b-4aa3-8b8b-be86f92cf7ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.479766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207d5731-a44b-4aa3-8b8b-be86f92cf7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.504272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-207d5731-a44b-4aa3-8b8b-be86f92cf7ac None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:40.505561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-207d5731-a44b-4aa3-8b8b-be86f92cf7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:40.510295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322114a-acaa-48af-8255-8a4dcc6f2ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:40.511262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322114a-acaa-48af-8255-8a4dcc6f2ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.511748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322114a-acaa-48af-8255-8a4dcc6f2ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.512199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4322114a-acaa-48af-8255-8a4dcc6f2ae9 None admin] PATH_INFO: `/v3/users/6cadfcaeb66a446fbb857f9049fb0196` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.534165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322114a-acaa-48af-8255-8a4dcc6f2ae9 None admin] RBAC: Authorizing `identity:get_user(user_id=6cadfcaeb66a446fbb857f9049fb0196)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:40.536123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4322114a-acaa-48af-8255-8a4dcc6f2ae9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:40.542860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 135/279] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:40 2022] GET /identity/v3/users/6cadfcaeb66a446fbb857f9049fb0196 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:40.544415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322114a-acaa-48af-8255-8a4dcc6f2ae9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:40.594455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:40.609663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 145/280] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:40.660963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4322114a-acaa-48af-8255-8a4dcc6f2ae9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:40.704328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85f171-ed21-45e6-8604-2d5c09e3f461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:40.705008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85f171-ed21-45e6-8604-2d5c09e3f461 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.705287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85f171-ed21-45e6-8604-2d5c09e3f461 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.705554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85f171-ed21-45e6-8604-2d5c09e3f461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.706354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85f171-ed21-45e6-8604-2d5c09e3f461 None admin] RBAC: Adding query filter params (role.id=7b4a508ded224dfc812c05d174d41511, user.id=6cadfcaeb66a446fbb857f9049fb0196, scope.system=all) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:40.706659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85f171-ed21-45e6-8604-2d5c09e3f461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:40.708159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85f171-ed21-45e6-8604-2d5c09e3f461 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:40.713988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 136/281] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Oct 5 13:57:40 2022] GET /identity/v3/role_assignments?user.id=6cadfcaeb66a446fbb857f9049fb0196&scope.system=all&role.id=7b4a508ded224dfc812c05d174d41511 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:40.789052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a1b1-3873-4e38-8c10-de4999c89ebf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.789365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a1b1-3873-4e38-8c10-de4999c89ebf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.789586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6076a1b1-3873-4e38-8c10-de4999c89ebf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.791280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 146/282] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:40.798140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e74da0-de67-4cf7-b8d3-453d4b21dcb3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:40.798435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e74da0-de67-4cf7-b8d3-453d4b21dcb3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:40.798685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e74da0-de67-4cf7-b8d3-453d4b21dcb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:40.821416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f4e74da0-de67-4cf7-b8d3-453d4b21dcb3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:40.822650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f4e74da0-de67-4cf7-b8d3-453d4b21dcb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:40.873604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a1b1-3873-4e38-8c10-de4999c89ebf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:40.904046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 137/283] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:41.013504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6076a1b1-3873-4e38-8c10-de4999c89ebf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:41.035238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:41.064194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2026d-5f59-48c2-a3c2-94e466c59a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:41.064844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2026d-5f59-48c2-a3c2-94e466c59a2d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:41.065123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2026d-5f59-48c2-a3c2-94e466c59a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:41.065393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2026d-5f59-48c2-a3c2-94e466c59a2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:41.079596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2026d-5f59-48c2-a3c2-94e466c59a2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9eb2026d-5f59-48c2-a3c2-94e466c59a2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:41.084470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:41.085508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:41.086320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:41.089989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 147/284] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:41.091554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e74da0-de67-4cf7-b8d3-453d4b21dcb3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:41.148505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:41.233414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e74da0-de67-4cf7-b8d3-453d4b21dcb3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:41.269598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:41.270272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:41.270529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:41.270768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:41.271435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:41.271728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:41.273076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:41.277097 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2026d-5f59-48c2-a3c2-94e466c59a2d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:41.285743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 138/285] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:41.294864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:41.448067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5689c8b6-f7ec-4977-b166-dc67564b221d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:41.452854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2026d-5f59-48c2-a3c2-94e466c59a2d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:41.460209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:41.491823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2abdf-967e-4cfc-804d-f81ae7b74d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:41.492829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2abdf-967e-4cfc-804d-f81ae7b74d9d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:41.493332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2abdf-967e-4cfc-804d-f81ae7b74d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:41.493827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa2abdf-967e-4cfc-804d-f81ae7b74d9d None admin] PATH_INFO: `/v3/users/1adcfd23be71456996bca91341df02a0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:41.495195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2abdf-967e-4cfc-804d-f81ae7b74d9d None admin] RBAC: Authorizing `identity:get_user(user_id=1adcfd23be71456996bca91341df02a0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:41.496835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa2abdf-967e-4cfc-804d-f81ae7b74d9d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:41.499056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 139/286] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:41 2022] GET /identity/v3/users/1adcfd23be71456996bca91341df02a0 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:41.511996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2abdf-967e-4cfc-804d-f81ae7b74d9d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:41.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856ddb9-0970-423b-9269-69c11f320d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:41.559608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856ddb9-0970-423b-9269-69c11f320d00 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:41.559889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856ddb9-0970-423b-9269-69c11f320d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:41.560160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0856ddb9-0970-423b-9269-69c11f320d00 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:41.595819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0856ddb9-0970-423b-9269-69c11f320d00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0856ddb9-0970-423b-9269-69c11f320d00 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:41.602701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:41.603623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:41.604410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:41.607406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 148/287] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:41.613544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856ddb9-0970-423b-9269-69c11f320d00 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:41.648570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa2abdf-967e-4cfc-804d-f81ae7b74d9d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:41.681992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:41.728326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718b728-5ab4-4955-b4bf-07061698a3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:41.729366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718b728-5ab4-4955-b4bf-07061698a3ff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:41.729892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718b728-5ab4-4955-b4bf-07061698a3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:41.730354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718b728-5ab4-4955-b4bf-07061698a3ff None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:41.731983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718b728-5ab4-4955-b4bf-07061698a3ff None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:41.733820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718b728-5ab4-4955-b4bf-07061698a3ff None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:41.740351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 140/288] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:41 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:41.744622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718b728-5ab4-4955-b4bf-07061698a3ff None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:41.788117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0856ddb9-0970-423b-9269-69c11f320d00 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:41.825075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:41.825995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:41.826735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:41.827224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:41.828156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:41.828651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:41.830381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:41.831989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:41.841597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 149/289] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:41.852331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:41.916280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718b728-5ab4-4955-b4bf-07061698a3ff None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:41.951190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad8d0f-b235-4ebc-b759-d3aaa3dc7ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:41.952001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad8d0f-b235-4ebc-b759-d3aaa3dc7ba5 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:41.952281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad8d0f-b235-4ebc-b759-d3aaa3dc7ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:41.952550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ad8d0f-b235-4ebc-b759-d3aaa3dc7ba5 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9/users/1adcfd23be71456996bca91341df02a0/roles/2e8c5a373cb5440d91307a8c8ab3186d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:41.954472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad8d0f-b235-4ebc-b759-d3aaa3dc7ba5 None admin] RBAC: Authorizing `identity:create_grant(project_id=bb88575700f64da982e71dffc4bac6e9, user_id=1adcfd23be71456996bca91341df02a0, role_id=2e8c5a373cb5440d91307a8c8ab3186d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:41.955895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ad8d0f-b235-4ebc-b759-d3aaa3dc7ba5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:41.980085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 141/290] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:57:41 2022] PUT /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9/users/1adcfd23be71456996bca91341df02a0/roles/2e8c5a373cb5440d91307a8c8ab3186d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:42.063954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:42.069092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a501d8ad-6cb6-423a-95d8-44e158e1021a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:42.133604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce31fad-7dc1-44d7-9532-ed32be32ae12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:42.134625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce31fad-7dc1-44d7-9532-ed32be32ae12 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:42.135124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce31fad-7dc1-44d7-9532-ed32be32ae12 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:42.135611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce31fad-7dc1-44d7-9532-ed32be32ae12 None admin] PATH_INFO: `/v3/users/b0f78890553e43e8ac446cbdab22c935` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:42.137131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce31fad-7dc1-44d7-9532-ed32be32ae12 None admin] RBAC: Authorizing `identity:get_user(user_id=b0f78890553e43e8ac446cbdab22c935)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:42.139393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce31fad-7dc1-44d7-9532-ed32be32ae12 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:42.141826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 150/291] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:41 2022] GET /identity/v3/users/b0f78890553e43e8ac446cbdab22c935 => generated 297 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:42.148205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad8d0f-b235-4ebc-b759-d3aaa3dc7ba5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:42.256860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:42.266170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ad8d0f-b235-4ebc-b759-d3aaa3dc7ba5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:42.310327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342be3fe-f11f-487f-b71d-94e56a779506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:42.310954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be3fe-f11f-487f-b71d-94e56a779506 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:42.311217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be3fe-f11f-487f-b71d-94e56a779506 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:42.311487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342be3fe-f11f-487f-b71d-94e56a779506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:42.410012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342be3fe-f11f-487f-b71d-94e56a779506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:42.411609 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342be3fe-f11f-487f-b71d-94e56a779506 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:42.551082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:42.561805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 142/292] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:42.565076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce31fad-7dc1-44d7-9532-ed32be32ae12 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:42.713006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce31fad-7dc1-44d7-9532-ed32be32ae12 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:42.753099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682b15e-88d3-4e4c-ab21-b1189b5ec212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:42.753776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b15e-88d3-4e4c-ab21-b1189b5ec212 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:42.754055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b15e-88d3-4e4c-ab21-b1189b5ec212 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:42.754328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682b15e-88d3-4e4c-ab21-b1189b5ec212 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:42.755482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b15e-88d3-4e4c-ab21-b1189b5ec212 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:42.756943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682b15e-88d3-4e4c-ab21-b1189b5ec212 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:42.758923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 151/293] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:42 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:42.764581 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342be3fe-f11f-487f-b71d-94e56a779506 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:42.913412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342be3fe-f11f-487f-b71d-94e56a779506 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:42.963487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4bc91d-8741-43a5-a3a2-5b7fa7123b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:42.964032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bc91d-8741-43a5-a3a2-5b7fa7123b6e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:42.964308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bc91d-8741-43a5-a3a2-5b7fa7123b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:42.964577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4bc91d-8741-43a5-a3a2-5b7fa7123b6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:42.965326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bc91d-8741-43a5-a3a2-5b7fa7123b6e None admin] RBAC: Adding query filter params (role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=bb88575700f64da982e71dffc4bac6e9, user.id=b0f78890553e43e8ac446cbdab22c935) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:42.965628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bc91d-8741-43a5-a3a2-5b7fa7123b6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:42.967120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4bc91d-8741-43a5-a3a2-5b7fa7123b6e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:42.974785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 143/294] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:42 2022] GET /identity/v3/role_assignments?user.id=b0f78890553e43e8ac446cbdab22c935&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=7b4a508ded224dfc812c05d174d41511 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:43.129077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b76c-602d-445a-bf13-ea6643649ac6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.129450 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b76c-602d-445a-bf13-ea6643649ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.129734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3138b76c-602d-445a-bf13-ea6643649ac6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.131521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 152/295] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:43.137490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d4d40-4b3e-419f-a125-eb0dacb0a3ca None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.137813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d4d40-4b3e-419f-a125-eb0dacb0a3ca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.138091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d4d40-4b3e-419f-a125-eb0dacb0a3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.163454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1f0d4d40-4b3e-419f-a125-eb0dacb0a3ca None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:43.164691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f0d4d40-4b3e-419f-a125-eb0dacb0a3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:43.242629 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:43.258727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 144/296] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:43.374577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d11900-d4cd-42cf-93e5-bbce5410c018 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.374849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d11900-d4cd-42cf-93e5-bbce5410c018 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.375121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d11900-d4cd-42cf-93e5-bbce5410c018 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.376759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 153/297] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:43.383316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8bae0-8987-45fe-9c1b-1beea9ba6a69 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.383777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8bae0-8987-45fe-9c1b-1beea9ba6a69 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.383850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd8bae0-8987-45fe-9c1b-1beea9ba6a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.404611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5fd8bae0-8987-45fe-9c1b-1beea9ba6a69 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:43.405817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5fd8bae0-8987-45fe-9c1b-1beea9ba6a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:43.484083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 145/298] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:43.546475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d11900-d4cd-42cf-93e5-bbce5410c018 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:43.589085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:43.661168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d11900-d4cd-42cf-93e5-bbce5410c018 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:43.681261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf27c48-ac69-4e66-a3e3-ad14c4a903a3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.681533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf27c48-ac69-4e66-a3e3-ad14c4a903a3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.681839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf27c48-ac69-4e66-a3e3-ad14c4a903a3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.683421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 146/299] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:57:43.688133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815174a-6eca-40e4-8f7d-f6170bf8fcb7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.688408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815174a-6eca-40e4-8f7d-f6170bf8fcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.688674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3815174a-6eca-40e4-8f7d-f6170bf8fcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.696037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936bdf7-3e35-44cd-b677-a64615887aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:43.696673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936bdf7-3e35-44cd-b677-a64615887aab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.696946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936bdf7-3e35-44cd-b677-a64615887aab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.697210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0936bdf7-3e35-44cd-b677-a64615887aab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.698316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936bdf7-3e35-44cd-b677-a64615887aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:43.699742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0936bdf7-3e35-44cd-b677-a64615887aab None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:43.701338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 154/300] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:43.707112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936bdf7-3e35-44cd-b677-a64615887aab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:43.707490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3815174a-6eca-40e4-8f7d-f6170bf8fcb7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:43.708689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3815174a-6eca-40e4-8f7d-f6170bf8fcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:43.778626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 147/301] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:43.779394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815174a-6eca-40e4-8f7d-f6170bf8fcb7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:43.815037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0936bdf7-3e35-44cd-b677-a64615887aab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:43.816655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:43.842017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:43.851958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6947d1-8261-4191-bcfb-13e073cfa544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:43.852672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6947d1-8261-4191-bcfb-13e073cfa544 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.852943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6947d1-8261-4191-bcfb-13e073cfa544 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.853208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6947d1-8261-4191-bcfb-13e073cfa544 None admin] PATH_INFO: `/v3/groups` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.853916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6947d1-8261-4191-bcfb-13e073cfa544 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:43.855467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6947d1-8261-4191-bcfb-13e073cfa544 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:43.881694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 155/302] 10.222.0.23 () {60 vars in 1134 bytes} [Wed Oct 5 13:57:43 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:57:43.905856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3815174a-6eca-40e4-8f7d-f6170bf8fcb7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:43.945873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fda31-415f-4977-b4c6-0d63efe825b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:43.946643 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fda31-415f-4977-b4c6-0d63efe825b3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:43.946917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fda31-415f-4977-b4c6-0d63efe825b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:43.947182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13fda31-415f-4977-b4c6-0d63efe825b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:43.960005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13fda31-415f-4977-b4c6-0d63efe825b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d13fda31-415f-4977-b4c6-0d63efe825b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:43.965044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:43.965955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:43.966759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 148/303] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:43.972534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6947d1-8261-4191-bcfb-13e073cfa544 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:44.064691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fda31-415f-4977-b4c6-0d63efe825b3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:44.092936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6947d1-8261-4191-bcfb-13e073cfa544 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:44.135975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:44.136950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.137419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.137945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.138829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:44.139326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:44.140929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:44.149983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 156/304] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:44.155602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:44.182407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:44.196229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13fda31-415f-4977-b4c6-0d63efe825b3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:44.207800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:44.239659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf6ef5-0b53-455b-955f-a8862bd5dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:44.240305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf6ef5-0b53-455b-955f-a8862bd5dbac None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.240575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf6ef5-0b53-455b-955f-a8862bd5dbac None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.240837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf6ef5-0b53-455b-955f-a8862bd5dbac None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.253368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf6ef5-0b53-455b-955f-a8862bd5dbac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5fbf6ef5-0b53-455b-955f-a8862bd5dbac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:44.259295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:44.260164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:44.261025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:44.265351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 149/305] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:44.267014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf6ef5-0b53-455b-955f-a8862bd5dbac None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:44.280187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6ead9-9871-44c3-87ba-a384428c39fd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:44.321859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf958d-803a-4af6-8fcb-f6040b1d305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:44.322418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf958d-803a-4af6-8fcb-f6040b1d305c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.322693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf958d-803a-4af6-8fcb-f6040b1d305c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.322963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf958d-803a-4af6-8fcb-f6040b1d305c None admin] PATH_INFO: `/v3/users/12c3d596c015416dbb0623443c9c543d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.339843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf958d-803a-4af6-8fcb-f6040b1d305c None admin] RBAC: Authorizing `identity:get_user(user_id=12c3d596c015416dbb0623443c9c543d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:44.341473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf958d-803a-4af6-8fcb-f6040b1d305c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:44.349043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 157/306] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:44 2022] GET /identity/v3/users/12c3d596c015416dbb0623443c9c543d => generated 293 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:44.351413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf958d-803a-4af6-8fcb-f6040b1d305c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:44.387236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf6ef5-0b53-455b-955f-a8862bd5dbac None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:44.431495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:44.432236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.432573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.432903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.433661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:44.434023 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:44.435630 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:44.449342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 150/307] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:44.455188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:44.458213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:44.480115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf958d-803a-4af6-8fcb-f6040b1d305c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:44.488725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:44.519919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f9cd6-3e6c-4b8e-9fd8-b84f39e30f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:44.520768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f9cd6-3e6c-4b8e-9fd8-b84f39e30f2c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.520960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f9cd6-3e6c-4b8e-9fd8-b84f39e30f2c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.521235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f9cd6-3e6c-4b8e-9fd8-b84f39e30f2c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.522006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f9cd6-3e6c-4b8e-9fd8-b84f39e30f2c None admin] RBAC: Adding query filter params (role.id=2e8c5a373cb5440d91307a8c8ab3186d, user.id=12c3d596c015416dbb0623443c9c543d, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:44.522312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f9cd6-3e6c-4b8e-9fd8-b84f39e30f2c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:44.523778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f9cd6-3e6c-4b8e-9fd8-b84f39e30f2c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:44.529489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 158/308] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Oct 5 13:57:44 2022] GET /identity/v3/role_assignments?user.id=12c3d596c015416dbb0623443c9c543d&scope.system=all&role.id=2e8c5a373cb5440d91307a8c8ab3186d => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:44.530718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b491e3-870a-4ed9-a7a9-a78d0054c194 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.530999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b491e3-870a-4ed9-a7a9-a78d0054c194 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.531270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b491e3-870a-4ed9-a7a9-a78d0054c194 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.533789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 159/309] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:44.538535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b0b12-f950-4ea8-b23d-8e76240ce1a1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.539005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b0b12-f950-4ea8-b23d-8e76240ce1a1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.539418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7b0b12-f950-4ea8-b23d-8e76240ce1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.560270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5e7b0b12-f950-4ea8-b23d-8e76240ce1a1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:44.561968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e7b0b12-f950-4ea8-b23d-8e76240ce1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:44.578889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c430f8-082a-4c94-bd3c-3ad96ff15c66 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:44.618900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6052b-b6a0-40fc-8814-46829cf7aa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:44.619731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6052b-b6a0-40fc-8814-46829cf7aa6d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.620144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6052b-b6a0-40fc-8814-46829cf7aa6d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.620508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c6052b-b6a0-40fc-8814-46829cf7aa6d None admin] PATH_INFO: `/v3/users/6cadfcaeb66a446fbb857f9049fb0196` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.621971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6052b-b6a0-40fc-8814-46829cf7aa6d None admin] RBAC: Authorizing `identity:get_user(user_id=6cadfcaeb66a446fbb857f9049fb0196)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:44.623717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c6052b-b6a0-40fc-8814-46829cf7aa6d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:44.625979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 151/310] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:44 2022] GET /identity/v3/users/6cadfcaeb66a446fbb857f9049fb0196 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:44.632304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6052b-b6a0-40fc-8814-46829cf7aa6d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:44.661086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 160/311] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:44.749841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:44.753930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c6052b-b6a0-40fc-8814-46829cf7aa6d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:44.789262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74e522-206f-4ac9-8585-ac28281d3b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:44.789969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e522-206f-4ac9-8585-ac28281d3b7a None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:44.790245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e522-206f-4ac9-8585-ac28281d3b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:44.790521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf74e522-206f-4ac9-8585-ac28281d3b7a None admin] PATH_INFO: `/v3/system/users/6cadfcaeb66a446fbb857f9049fb0196/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:44.791906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74e522-206f-4ac9-8585-ac28281d3b7a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6cadfcaeb66a446fbb857f9049fb0196, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:44.793368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf74e522-206f-4ac9-8585-ac28281d3b7a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:44.809822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 152/312] 10.222.0.23 () {58 vars in 1334 bytes} [Wed Oct 5 13:57:44 2022] PUT /identity/v3/system/users/6cadfcaeb66a446fbb857f9049fb0196/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 13:57:44.930068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b0b12-f950-4ea8-b23d-8e76240ce1a1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:45.015752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:45.043551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7b0b12-f950-4ea8-b23d-8e76240ce1a1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:45.079258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24a881-1e50-4f98-a963-6978ad4f64a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:45.080255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a881-1e50-4f98-a963-6978ad4f64a0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:45.080748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a881-1e50-4f98-a963-6978ad4f64a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:45.081231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af24a881-1e50-4f98-a963-6978ad4f64a0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:45.091996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af24a881-1e50-4f98-a963-6978ad4f64a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-af24a881-1e50-4f98-a963-6978ad4f64a0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:45.097071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:45.097982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:45.098995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:45.100892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 161/313] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:45.107230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74e522-206f-4ac9-8585-ac28281d3b7a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:45.211521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf74e522-206f-4ac9-8585-ac28281d3b7a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:45.249420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:45.250304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:45.250758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:45.251160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:45.251959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:45.252392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:45.253942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:45.262507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 153/314] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:45.271588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24a881-1e50-4f98-a963-6978ad4f64a0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:45.378890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af24a881-1e50-4f98-a963-6978ad4f64a0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:45.415105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419bbb9-f308-4348-b206-4bd39b4df249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:45.415759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419bbb9-f308-4348-b206-4bd39b4df249 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:45.416035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419bbb9-f308-4348-b206-4bd39b4df249 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:45.416309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419bbb9-f308-4348-b206-4bd39b4df249 None admin] PATH_INFO: `/v3/users/1adcfd23be71456996bca91341df02a0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:45.417463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419bbb9-f308-4348-b206-4bd39b4df249 None admin] RBAC: Authorizing `identity:get_user(user_id=1adcfd23be71456996bca91341df02a0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:45.419002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419bbb9-f308-4348-b206-4bd39b4df249 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:45.420954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 162/315] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:45 2022] GET /identity/v3/users/1adcfd23be71456996bca91341df02a0 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:45.426749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:45.452843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:45.545130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dcf260-1d8c-4665-8834-8ea050718376 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:45.578789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c2a53-2ffb-4f35-9a7a-f3098b5ad45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:45.579414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c2a53-2ffb-4f35-9a7a-f3098b5ad45a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:45.579686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c2a53-2ffb-4f35-9a7a-f3098b5ad45a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:45.579951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c2a53-2ffb-4f35-9a7a-f3098b5ad45a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:45.646932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c2a53-2ffb-4f35-9a7a-f3098b5ad45a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:45.648435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c2a53-2ffb-4f35-9a7a-f3098b5ad45a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:45.672105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:45.749529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 154/316] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:45.755003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419bbb9-f308-4348-b206-4bd39b4df249 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:45.759206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:45.860968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419bbb9-f308-4348-b206-4bd39b4df249 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:45.896489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658a5afc-e99c-4f88-8c06-8488bedab76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:45.897535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a5afc-e99c-4f88-8c06-8488bedab76a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:45.898053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a5afc-e99c-4f88-8c06-8488bedab76a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:45.898515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658a5afc-e99c-4f88-8c06-8488bedab76a None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:45.899828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a5afc-e99c-4f88-8c06-8488bedab76a None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:45.901487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658a5afc-e99c-4f88-8c06-8488bedab76a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:45.903649 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 163/317] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:45 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:45.909817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c2a53-2ffb-4f35-9a7a-f3098b5ad45a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:46.011079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c2a53-2ffb-4f35-9a7a-f3098b5ad45a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:46.038813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:46.050751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e0377-c4ef-4c31-ae6e-8270ffa70c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:46.052066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e0377-c4ef-4c31-ae6e-8270ffa70c22 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:46.052346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e0377-c4ef-4c31-ae6e-8270ffa70c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:46.052621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e0377-c4ef-4c31-ae6e-8270ffa70c22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:46.053368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e0377-c4ef-4c31-ae6e-8270ffa70c22 None admin] RBAC: Adding query filter params (role.id=2e8c5a373cb5440d91307a8c8ab3186d, scope.project.id=bb88575700f64da982e71dffc4bac6e9, user.id=1adcfd23be71456996bca91341df02a0) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:46.053709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e0377-c4ef-4c31-ae6e-8270ffa70c22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:46.055284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e0377-c4ef-4c31-ae6e-8270ffa70c22 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:46.062588 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 155/318] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:57:45 2022] GET /identity/v3/role_assignments?user.id=1adcfd23be71456996bca91341df02a0&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=2e8c5a373cb5440d91307a8c8ab3186d => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:46.161303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ecdc7-9d1e-4a5b-a08c-8edbbdc99530 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:46.161990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ecdc7-9d1e-4a5b-a08c-8edbbdc99530 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:46.162489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027ecdc7-9d1e-4a5b-a08c-8edbbdc99530 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:46.164573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 164/319] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:46.170738 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e63c6-46b1-4dc5-960a-2809ad14b91a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:46.171294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e63c6-46b1-4dc5-960a-2809ad14b91a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:46.171750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e63c6-46b1-4dc5-960a-2809ad14b91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:46.191217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-414e63c6-46b1-4dc5-960a-2809ad14b91a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:46.192852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-414e63c6-46b1-4dc5-960a-2809ad14b91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:46.270812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 156/320] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:46.539450 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ecdc7-9d1e-4a5b-a08c-8edbbdc99530 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:46.616308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:46.648736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027ecdc7-9d1e-4a5b-a08c-8edbbdc99530 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:46.684506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79faaf3-06ba-4a45-8953-ab2730a85c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:46.685148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79faaf3-06ba-4a45-8953-ab2730a85c86 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:46.685426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79faaf3-06ba-4a45-8953-ab2730a85c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:46.685702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79faaf3-06ba-4a45-8953-ab2730a85c86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:46.686859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79faaf3-06ba-4a45-8953-ab2730a85c86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:46.688463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79faaf3-06ba-4a45-8953-ab2730a85c86 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:46.690308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 165/321] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:57:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:46.696056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e63c6-46b1-4dc5-960a-2809ad14b91a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:46.801002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e63c6-46b1-4dc5-960a-2809ad14b91a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:46.818310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:46.823820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cb75-2995-4860-8363-ca5c86dae4fe None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:46.824296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cb75-2995-4860-8363-ca5c86dae4fe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:46.824398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7433cb75-2995-4860-8363-ca5c86dae4fe None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:46.825883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 166/322] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:57:46.830589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140159-eeb3-4770-b9c5-1d2438ca1318 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:46.830859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140159-eeb3-4770-b9c5-1d2438ca1318 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:46.831138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e140159-eeb3-4770-b9c5-1d2438ca1318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:46.837701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362e8ba-78fa-476d-b906-9db9453468dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:46.838315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e8ba-78fa-476d-b906-9db9453468dc None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:46.838595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e8ba-78fa-476d-b906-9db9453468dc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:46.838869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362e8ba-78fa-476d-b906-9db9453468dc None admin] PATH_INFO: `/v3/groups` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:46.839539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362e8ba-78fa-476d-b906-9db9453468dc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:46.840971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362e8ba-78fa-476d-b906-9db9453468dc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:46.849912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5e140159-eeb3-4770-b9c5-1d2438ca1318 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:46.851115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5e140159-eeb3-4770-b9c5-1d2438ca1318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:46.858195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 157/323] 10.222.0.23 () {60 vars in 1134 bytes} [Wed Oct 5 13:57:46 2022] POST /identity/v3/groups => generated 226 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:57:46.928888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 167/324] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:57:47.016247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d69e-0108-492a-8e46-93e1e495a985 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:47.016604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d69e-0108-492a-8e46-93e1e495a985 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:47.016756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a9d69e-0108-492a-8e46-93e1e495a985 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:47.018394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 158/325] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:47.024122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bee68-5f5b-4b10-bf6c-c7f7d5891621 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:47.024440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bee68-5f5b-4b10-bf6c-c7f7d5891621 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:47.024697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3bee68-5f5b-4b10-bf6c-c7f7d5891621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:47.044139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dc3bee68-5f5b-4b10-bf6c-c7f7d5891621 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:47.045306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc3bee68-5f5b-4b10-bf6c-c7f7d5891621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:47.109759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:47.115645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 168/326] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:47.190491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9d69e-0108-492a-8e46-93e1e495a985 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:47.296061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a9d69e-0108-492a-8e46-93e1e495a985 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:47.334999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938f417-1acc-4281-9dea-5e92834c6991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:47.335650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938f417-1acc-4281-9dea-5e92834c6991 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:47.335922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938f417-1acc-4281-9dea-5e92834c6991 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:47.336190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a938f417-1acc-4281-9dea-5e92834c6991 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:47.346969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a938f417-1acc-4281-9dea-5e92834c6991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a938f417-1acc-4281-9dea-5e92834c6991 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:47.353248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:47.354322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:47.355797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 159/327] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:47.360914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bee68-5f5b-4b10-bf6c-c7f7d5891621 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:47.385013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938f417-1acc-4281-9dea-5e92834c6991 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:47.431531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:47.503068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3bee68-5f5b-4b10-bf6c-c7f7d5891621 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:47.519645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a938f417-1acc-4281-9dea-5e92834c6991 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:47.550102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:47.551446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:47.552123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:47.552534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:47.553660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:47.554119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:47.556237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:47.563227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618362ea-f775-483c-84e3-4f84b6033b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:47.564284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618362ea-f775-483c-84e3-4f84b6033b0d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:47.564681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618362ea-f775-483c-84e3-4f84b6033b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:47.565071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618362ea-f775-483c-84e3-4f84b6033b0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:47.566498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 169/328] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:47.575079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:47.584675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618362ea-f775-483c-84e3-4f84b6033b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-618362ea-f775-483c-84e3-4f84b6033b0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:47.591294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:47.592148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:47.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 160/329] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:47.600611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618362ea-f775-483c-84e3-4f84b6033b0d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:47.661364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:47.715717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b2af78-bc4d-4d71-a710-67531eab5705 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:47.734555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:47.739037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618362ea-f775-483c-84e3-4f84b6033b0d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:47.763856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251581dc-b20e-45fc-94fa-8b55e5cfb368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:47.764510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251581dc-b20e-45fc-94fa-8b55e5cfb368 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:47.764786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251581dc-b20e-45fc-94fa-8b55e5cfb368 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:47.765052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251581dc-b20e-45fc-94fa-8b55e5cfb368 None admin] PATH_INFO: `/v3/users/12c3d596c015416dbb0623443c9c543d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:47.767039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251581dc-b20e-45fc-94fa-8b55e5cfb368 None admin] RBAC: Authorizing `identity:get_user(user_id=12c3d596c015416dbb0623443c9c543d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:47.768516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251581dc-b20e-45fc-94fa-8b55e5cfb368 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:47.770269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 170/330] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:47 2022] GET /identity/v3/users/12c3d596c015416dbb0623443c9c543d => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:47.775855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251581dc-b20e-45fc-94fa-8b55e5cfb368 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:47.800517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:47.801144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:47.801475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:47.801778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:47.802457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:47.802760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:47.804155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:47.817689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 161/331] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:47.818709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:47.926488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec3a066-5835-40fb-81a6-c4b554df4a35 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:47.936785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251581dc-b20e-45fc-94fa-8b55e5cfb368 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:47.962423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86706ac4-a7c6-498b-b88f-c43f87cd934d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:47.962423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86706ac4-a7c6-498b-b88f-c43f87cd934d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:47.962423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86706ac4-a7c6-498b-b88f-c43f87cd934d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:47.962423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86706ac4-a7c6-498b-b88f-c43f87cd934d None admin] PATH_INFO: `/v3/users/6cadfcaeb66a446fbb857f9049fb0196` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:47.963444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86706ac4-a7c6-498b-b88f-c43f87cd934d None admin] RBAC: Authorizing `identity:get_user(user_id=6cadfcaeb66a446fbb857f9049fb0196)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:47.964885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86706ac4-a7c6-498b-b88f-c43f87cd934d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:47.966782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 162/332] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:47 2022] GET /identity/v3/users/6cadfcaeb66a446fbb857f9049fb0196 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:47.972410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86706ac4-a7c6-498b-b88f-c43f87cd934d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:47.995208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:48.011627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1fdfee-3388-4bf0-961d-48789577857c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:48.012297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdfee-3388-4bf0-961d-48789577857c None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:48.012570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdfee-3388-4bf0-961d-48789577857c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:48.012839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1fdfee-3388-4bf0-961d-48789577857c None admin] PATH_INFO: `/v3/system/users/12c3d596c015416dbb0623443c9c543d/roles/2e8c5a373cb5440d91307a8c8ab3186d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:48.018228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:48.018380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fdfee-3388-4bf0-961d-48789577857c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=12c3d596c015416dbb0623443c9c543d, role_id=2e8c5a373cb5440d91307a8c8ab3186d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:48.019916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1fdfee-3388-4bf0-961d-48789577857c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:48.035451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 171/333] 10.222.0.23 () {58 vars in 1334 bytes} [Wed Oct 5 13:57:47 2022] PUT /identity/v3/system/users/12c3d596c015416dbb0623443c9c543d/roles/2e8c5a373cb5440d91307a8c8ab3186d => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 13:57:48.085122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86706ac4-a7c6-498b-b88f-c43f87cd934d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:48.122884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbe67e1-b04f-4329-85cd-8b6e0006e39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:48.123835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe67e1-b04f-4329-85cd-8b6e0006e39b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:48.124280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe67e1-b04f-4329-85cd-8b6e0006e39b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:48.124723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbe67e1-b04f-4329-85cd-8b6e0006e39b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:48.125620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe67e1-b04f-4329-85cd-8b6e0006e39b None admin] RBAC: Adding query filter params (role.id=7b4a508ded224dfc812c05d174d41511, user.id=6cadfcaeb66a446fbb857f9049fb0196, scope.system=all) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:48.126116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe67e1-b04f-4329-85cd-8b6e0006e39b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:48.127820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbe67e1-b04f-4329-85cd-8b6e0006e39b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:48.141278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 163/334] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Oct 5 13:57:47 2022] GET /identity/v3/role_assignments?user.id=6cadfcaeb66a446fbb857f9049fb0196&scope.system=all&role.id=7b4a508ded224dfc812c05d174d41511 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:49.758126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4360a-0361-4ecc-8a9f-e1ba6d172463 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:49.759165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4360a-0361-4ecc-8a9f-e1ba6d172463 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:49.759734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4360a-0361-4ecc-8a9f-e1ba6d172463 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:49.761564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 172/335] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:49.767696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510ab0a-b318-4d54-827d-f789ca246b4d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:49.768573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510ab0a-b318-4d54-827d-f789ca246b4d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:49.769063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a510ab0a-b318-4d54-827d-f789ca246b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:49.789306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a510ab0a-b318-4d54-827d-f789ca246b4d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:49.790846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a510ab0a-b318-4d54-827d-f789ca246b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:49.881594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 164/336] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:50.020353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd81e1-e618-4b22-aac1-d5e72eb8f3ee None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:50.020962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd81e1-e618-4b22-aac1-d5e72eb8f3ee None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:50.021441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abd81e1-e618-4b22-aac1-d5e72eb8f3ee None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:50.023291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 173/337] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:50.039902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353436a-9426-4430-83cf-879d4a9c4d19 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:50.040658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353436a-9426-4430-83cf-879d4a9c4d19 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:50.041408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5353436a-9426-4430-83cf-879d4a9c4d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:50.066772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5353436a-9426-4430-83cf-879d4a9c4d19 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:50.068313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5353436a-9426-4430-83cf-879d4a9c4d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:50.085817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:50.151816 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 165/338] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:50.220535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd81e1-e618-4b22-aac1-d5e72eb8f3ee None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:50.351547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abd81e1-e618-4b22-aac1-d5e72eb8f3ee None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:50.397198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b612f-04ff-46ec-9b9e-84821f75f7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:50.397856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b612f-04ff-46ec-9b9e-84821f75f7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:50.398134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b612f-04ff-46ec-9b9e-84821f75f7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:50.398404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b612f-04ff-46ec-9b9e-84821f75f7a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:50.414995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b612f-04ff-46ec-9b9e-84821f75f7a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7f6b612f-04ff-46ec-9b9e-84821f75f7a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:50.419896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:50.420753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:50.421974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 174/339] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:50.430813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353436a-9426-4430-83cf-879d4a9c4d19 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:50.567868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b612f-04ff-46ec-9b9e-84821f75f7a4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:50.576195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5353436a-9426-4430-83cf-879d4a9c4d19 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:50.623627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:50.624170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:50.624496 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:50.624766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:50.625539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:50.626006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:50.627520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:50.631761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:50.638234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 166/340] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:50.644152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:50.662986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:50.691065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b612f-04ff-46ec-9b9e-84821f75f7a4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:50.730625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f6186-f10e-4da5-9d73-19d600769f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:50.731652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f6186-f10e-4da5-9d73-19d600769f01 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:50.732217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f6186-f10e-4da5-9d73-19d600769f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:50.732680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f6186-f10e-4da5-9d73-19d600769f01 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:50.745277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809f6186-f10e-4da5-9d73-19d600769f01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-809f6186-f10e-4da5-9d73-19d600769f01 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:50.750646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:50.751524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:50.752310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:50.755201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 175/341] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:50.762269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f6186-f10e-4da5-9d73-19d600769f01 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:50.790144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2f7dd5-763b-4c00-82fd-da4f7fd19f8d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:50.839829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14043afe-365f-49f7-8dd4-2bb2994ce644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:50.840457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043afe-365f-49f7-8dd4-2bb2994ce644 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:50.840732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043afe-365f-49f7-8dd4-2bb2994ce644 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:50.840998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14043afe-365f-49f7-8dd4-2bb2994ce644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:50.873609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f6186-f10e-4da5-9d73-19d600769f01 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:50.889553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:50.913275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:50.914361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:50.914838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:50.915300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:50.916164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:50.916658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:50.918301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:50.928380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 176/342] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:50.934384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:50.939628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14043afe-365f-49f7-8dd4-2bb2994ce644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:50.941097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14043afe-365f-49f7-8dd4-2bb2994ce644 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:50.956021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:51.065210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc75bb9-24c1-46b7-9999-1b4191c52979 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:51.094169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 167/343] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 13:57:51.096255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4c5b7-072e-4f32-a1e9-d5c9550111c3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.097141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4c5b7-072e-4f32-a1e9-d5c9550111c3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.097867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e4c5b7-072e-4f32-a1e9-d5c9550111c3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.099743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 168/344] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:51.100696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4c5b7-072e-4f32-a1e9-d5c9550111c3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:51.107648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55b25b-50c4-47b9-92fe-17f35b4077ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:51.108329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55b25b-50c4-47b9-92fe-17f35b4077ea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.108604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55b25b-50c4-47b9-92fe-17f35b4077ea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.108918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55b25b-50c4-47b9-92fe-17f35b4077ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.174986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:51.211081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55b25b-50c4-47b9-92fe-17f35b4077ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:51.213411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55b25b-50c4-47b9-92fe-17f35b4077ea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:51.278770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e4c5b7-072e-4f32-a1e9-d5c9550111c3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:51.296055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:51.328793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a6d77-74ba-4b8f-a8d0-37ef4e1ab9ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:51.330264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a6d77-74ba-4b8f-a8d0-37ef4e1ab9ef None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.330732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a6d77-74ba-4b8f-a8d0-37ef4e1ab9ef None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.331184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a6d77-74ba-4b8f-a8d0-37ef4e1ab9ef None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.332579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a6d77-74ba-4b8f-a8d0-37ef4e1ab9ef None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:51.334251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a6d77-74ba-4b8f-a8d0-37ef4e1ab9ef None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:51.336449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 169/345] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:51 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:51.337829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a9b5-b191-457f-bc30-7052a142fa22 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.338314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a9b5-b191-457f-bc30-7052a142fa22 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.338766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d042a9b5-b191-457f-bc30-7052a142fa22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.366549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d042a9b5-b191-457f-bc30-7052a142fa22 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:51.368094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d042a9b5-b191-457f-bc30-7052a142fa22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:51.431602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 177/346] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 13:57:51.432802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55b25b-50c4-47b9-92fe-17f35b4077ea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:51.454130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 170/347] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:51.455684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5816-1228-4013-9b8d-2f32c762d5c5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.456097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5816-1228-4013-9b8d-2f32c762d5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.456494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e5816-1228-4013-9b8d-2f32c762d5c5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.458205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 171/348] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:51.459131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e5816-1228-4013-9b8d-2f32c762d5c5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:51.560346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:51.592953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55b25b-50c4-47b9-92fe-17f35b4077ea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:51.603254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e5816-1228-4013-9b8d-2f32c762d5c5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:51.614070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:51.644162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f305fc8-22d9-4247-837b-f156c0275fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:51.644812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305fc8-22d9-4247-837b-f156c0275fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.645316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305fc8-22d9-4247-837b-f156c0275fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.645803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f305fc8-22d9-4247-837b-f156c0275fc4 None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.650427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c9f30-48ff-45ed-8aaf-7f4fa2a9530c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:51.651346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c9f30-48ff-45ed-8aaf-7f4fa2a9530c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.651939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c9f30-48ff-45ed-8aaf-7f4fa2a9530c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.652369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c9f30-48ff-45ed-8aaf-7f4fa2a9530c None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.653728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c9f30-48ff-45ed-8aaf-7f4fa2a9530c None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:51.654193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f305fc8-22d9-4247-837b-f156c0275fc4 None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:51.655618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c9f30-48ff-45ed-8aaf-7f4fa2a9530c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:51.655733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f305fc8-22d9-4247-837b-f156c0275fc4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:51.657779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 172/350] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:51 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:51.657883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 178/350] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:51 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:51.659043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fdfce-9698-4c70-acf7-ee8f89e112ef None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.659316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fdfce-9698-4c70-acf7-ee8f89e112ef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.659583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003fdfce-9698-4c70-acf7-ee8f89e112ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.659858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe932e-b497-48ed-aedb-a5b2d6e87b5c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.660306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe932e-b497-48ed-aedb-a5b2d6e87b5c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.660815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe932e-b497-48ed-aedb-a5b2d6e87b5c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.666765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 173/351] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:51.667821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe932e-b497-48ed-aedb-a5b2d6e87b5c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:51.687170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-003fdfce-9698-4c70-acf7-ee8f89e112ef None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:51.688573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-003fdfce-9698-4c70-acf7-ee8f89e112ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:51.775479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 179/352] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:51.776358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fdfce-9698-4c70-acf7-ee8f89e112ef None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:51.799431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe932e-b497-48ed-aedb-a5b2d6e87b5c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:51.842503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc2df9d-4abe-4121-9bfc-d3bbd4a4a69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:51.843249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2df9d-4abe-4121-9bfc-d3bbd4a4a69c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.843509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2df9d-4abe-4121-9bfc-d3bbd4a4a69c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.843772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc2df9d-4abe-4121-9bfc-d3bbd4a4a69c None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.844965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2df9d-4abe-4121-9bfc-d3bbd4a4a69c None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:51.846436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc2df9d-4abe-4121-9bfc-d3bbd4a4a69c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:51.848691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 174/353] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:51 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:51.850547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb87501-cc06-4572-83b4-546f2dd692e7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.850822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb87501-cc06-4572-83b4-546f2dd692e7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.851086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb87501-cc06-4572-83b4-546f2dd692e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.852960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:51.872756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ebb87501-cc06-4572-83b4-546f2dd692e7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:51.874351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ebb87501-cc06-4572-83b4-546f2dd692e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:51.893650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003fdfce-9698-4c70-acf7-ee8f89e112ef None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:51.926827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:51.945935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:51.946622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:51.946963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:51.947296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:51.948093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] RBAC: Adding query filter params (group.id=98e2773c36d4495da03871c9871b770b, role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=d9df1c54939a4373af874595595358c5) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:51.948456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:51.950644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:51.958421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 180/354] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:57:51 2022] GET /identity/v3/role_assignments?group.id=98e2773c36d4495da03871c9871b770b&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=5491e6c9891843b8979a054e900d2e34 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:51.961011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:51.979928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 175/355] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:51.981219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb87501-cc06-4572-83b4-546f2dd692e7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:52.165673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb87501-cc06-4572-83b4-546f2dd692e7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:52.172006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931b9b7-333e-49cf-be95-ff5fa692dfce None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:52.204790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24af01-dafd-4295-9df9-94421bd11957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:52.205803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24af01-dafd-4295-9df9-94421bd11957 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:52.206251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24af01-dafd-4295-9df9-94421bd11957 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:52.206685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24af01-dafd-4295-9df9-94421bd11957 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:52.218515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24af01-dafd-4295-9df9-94421bd11957 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5e24af01-dafd-4295-9df9-94421bd11957 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:52.223726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:52.224645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:52.225453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:52.227803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 176/356] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:52.228777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24af01-dafd-4295-9df9-94421bd11957 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:52.260730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:52.261311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:52.261587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:52.261885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:52.262615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] RBAC: Adding query filter params (group.id=98e2773c36d4495da03871c9871b770b, role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=d9df1c54939a4373af874595595358c5) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:52.262917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:52.264435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:52.266518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:52.267101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:52.277138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 181/357] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:57:51 2022] GET /identity/v3/role_assignments?group.id=98e2773c36d4495da03871c9871b770b&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=7b4a508ded224dfc812c05d174d41511 => generated 260 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:52.277862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:52.349175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24af01-dafd-4295-9df9-94421bd11957 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:52.425558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269f37c-faff-4a05-b32b-ce1594ae280f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:52.426664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269f37c-faff-4a05-b32b-ce1594ae280f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:52.427119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269f37c-faff-4a05-b32b-ce1594ae280f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:52.427555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269f37c-faff-4a05-b32b-ce1594ae280f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:52.447591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9269f37c-faff-4a05-b32b-ce1594ae280f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9269f37c-faff-4a05-b32b-ce1594ae280f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:52.456150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:52.457073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:52.457949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:52.461248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 177/358] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:52.463906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269f37c-faff-4a05-b32b-ce1594ae280f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:52.482197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5277aa-e3ab-4150-a3f5-1c7ab3d62617 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:52.534028 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:52.535257 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:52.535750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:52.536188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:52.537576 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:52.538129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:52.540005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:52.551220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 182/359] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:52.552648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:52.626025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:52.642448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269f37c-faff-4a05-b32b-ce1594ae280f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:52.675870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:52.688623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea0f77-9be1-43f8-a4a4-0953252323f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:52.689279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0f77-9be1-43f8-a4a4-0953252323f4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:52.690230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0f77-9be1-43f8-a4a4-0953252323f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:52.690466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ea0f77-9be1-43f8-a4a4-0953252323f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:52.704153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ea0f77-9be1-43f8-a4a4-0953252323f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b5ea0f77-9be1-43f8-a4a4-0953252323f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:52.710862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:52.712014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:52.712986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 178/360] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:52.713807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea0f77-9be1-43f8-a4a4-0953252323f4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:52.730852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd0d83-1af4-43a6-abf6-f3af62481e7e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:52.799290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:52.799948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:52.800183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:52.801263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:52.801983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:52.802281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:52.803707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:52.822557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 183/361] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:52.823414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:52.843527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ea0f77-9be1-43f8-a4a4-0953252323f4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:52.883737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a27b9-ae9a-46e7-88b0-9bb201b15d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:52.884371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a27b9-ae9a-46e7-88b0-9bb201b15d87 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:52.884650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a27b9-ae9a-46e7-88b0-9bb201b15d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:52.884924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a27b9-ae9a-46e7-88b0-9bb201b15d87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:52.912316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:52.951634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f561ad-034a-404e-86a4-fbdac85ba3d3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:52.964634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a27b9-ae9a-46e7-88b0-9bb201b15d87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:52.966692 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a27b9-ae9a-46e7-88b0-9bb201b15d87 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:52.976779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:52.993410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d175823-0875-41b2-ad68-bfa5e6f8d28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:52.994091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d175823-0875-41b2-ad68-bfa5e6f8d28b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:52.994370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d175823-0875-41b2-ad68-bfa5e6f8d28b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:52.994640 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d175823-0875-41b2-ad68-bfa5e6f8d28b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:52.995458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d175823-0875-41b2-ad68-bfa5e6f8d28b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:52.995763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d175823-0875-41b2-ad68-bfa5e6f8d28b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:52.997219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d175823-0875-41b2-ad68-bfa5e6f8d28b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:53.007542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 184/362] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:53.012595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc36da2-7fa2-44cd-9af3-363828e4332a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:53.012595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc36da2-7fa2-44cd-9af3-363828e4332a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:53.012595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc36da2-7fa2-44cd-9af3-363828e4332a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:53.012595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 185/363] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:53.012595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc36da2-7fa2-44cd-9af3-363828e4332a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:53.105535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 179/364] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:53.106758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a27b9-ae9a-46e7-88b0-9bb201b15d87 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:53.142323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc36da2-7fa2-44cd-9af3-363828e4332a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:53.175421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:53.189265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7437b-d25b-42b6-b27e-d8310409e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:53.190067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7437b-d25b-42b6-b27e-d8310409e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:53.190593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7437b-d25b-42b6-b27e-d8310409e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:53.190961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7437b-d25b-42b6-b27e-d8310409e6a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:53.240864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a27b9-ae9a-46e7-88b0-9bb201b15d87 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:53.260927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:53.282889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7437b-d25b-42b6-b27e-d8310409e6a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:53.284746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be7437b-d25b-42b6-b27e-d8310409e6a6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:53.285376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778454e2-bcb4-4c1b-968d-0b2cf6dbb02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:53.286277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778454e2-bcb4-4c1b-968d-0b2cf6dbb02e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:53.286691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778454e2-bcb4-4c1b-968d-0b2cf6dbb02e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:53.287143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778454e2-bcb4-4c1b-968d-0b2cf6dbb02e None admin] PATH_INFO: `/v3/users/12c3d596c015416dbb0623443c9c543d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:53.290452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778454e2-bcb4-4c1b-968d-0b2cf6dbb02e None admin] RBAC: Authorizing `identity:get_user(user_id=12c3d596c015416dbb0623443c9c543d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:53.291925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778454e2-bcb4-4c1b-968d-0b2cf6dbb02e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:53.293730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 180/365] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:57:53 2022] GET /identity/v3/users/12c3d596c015416dbb0623443c9c543d => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:53.296613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97c747-a438-43e6-bad2-bd8486ae1e62 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:53.296613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97c747-a438-43e6-bad2-bd8486ae1e62 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:53.296613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97c747-a438-43e6-bad2-bd8486ae1e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:53.325599 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1e97c747-a438-43e6-bad2-bd8486ae1e62 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:53.326988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1e97c747-a438-43e6-bad2-bd8486ae1e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:53.409385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 181/366] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:53.411027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97c747-a438-43e6-bad2-bd8486ae1e62 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:53.430780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 186/367] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:57:53.431689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7437b-d25b-42b6-b27e-d8310409e6a6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:53.520886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:53.594396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:53.613300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be7437b-d25b-42b6-b27e-d8310409e6a6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:53.614682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97c747-a438-43e6-bad2-bd8486ae1e62 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:53.657741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1057ab4-1132-4415-ac91-dd20bf532bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:53.658358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1057ab4-1132-4415-ac91-dd20bf532bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:53.658634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1057ab4-1132-4415-ac91-dd20bf532bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:53.658903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1057ab4-1132-4415-ac91-dd20bf532bb3 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:53.662081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1057ab4-1132-4415-ac91-dd20bf532bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:53.663919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1057ab4-1132-4415-ac91-dd20bf532bb3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:53.666190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 182/368] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:53 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:53.666961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1057ab4-1132-4415-ac91-dd20bf532bb3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:53.669098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:53.669995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:53.670554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:53.671016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:53.671924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] RBAC: Adding query filter params (role.id=2e8c5a373cb5440d91307a8c8ab3186d, user.id=12c3d596c015416dbb0623443c9c543d, scope.system=all) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:53.672429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:53.674076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:53.701120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 187/369] 10.222.0.23 () {56 vars in 1308 bytes} [Wed Oct 5 13:57:53 2022] GET /identity/v3/role_assignments?user.id=12c3d596c015416dbb0623443c9c543d&scope.system=all&role.id=2e8c5a373cb5440d91307a8c8ab3186d => generated 510 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:53.702314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:53.779020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1057ab4-1132-4415-ac91-dd20bf532bb3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:53.802035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:53.817573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc35bce-b287-4fa9-a47a-c9f88e9488ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:53.818265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc35bce-b287-4fa9-a47a-c9f88e9488ae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:53.818539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc35bce-b287-4fa9-a47a-c9f88e9488ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:53.818806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc35bce-b287-4fa9-a47a-c9f88e9488ae None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:53.825658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc35bce-b287-4fa9-a47a-c9f88e9488ae None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:53.825658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc35bce-b287-4fa9-a47a-c9f88e9488ae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:53.825658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 183/370] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:53 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:53.831375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc35bce-b287-4fa9-a47a-c9f88e9488ae None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:53.966297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002be4f2-3795-4583-b14c-47293c411452 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:53.998229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc35bce-b287-4fa9-a47a-c9f88e9488ae None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:54.037832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4d331-5e8b-4f80-83d6-c97b152db9b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:54.038409 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4d331-5e8b-4f80-83d6-c97b152db9b0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.038683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4d331-5e8b-4f80-83d6-c97b152db9b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.038949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4d331-5e8b-4f80-83d6-c97b152db9b0 None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.044383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4d331-5e8b-4f80-83d6-c97b152db9b0 None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:54.045963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4d331-5e8b-4f80-83d6-c97b152db9b0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:54.048587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 184/371] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:53 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:54.049438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4d331-5e8b-4f80-83d6-c97b152db9b0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:54.057468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fc689-f622-405d-9bc5-355f8181aa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:54.058468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fc689-f622-405d-9bc5-355f8181aa6a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.058931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fc689-f622-405d-9bc5-355f8181aa6a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.059382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593fc689-f622-405d-9bc5-355f8181aa6a None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.060692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fc689-f622-405d-9bc5-355f8181aa6a None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:54.062350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593fc689-f622-405d-9bc5-355f8181aa6a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:54.064167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 188/372] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:53 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:54.065137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fc689-f622-405d-9bc5-355f8181aa6a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:54.091978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:54.126549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:54.170359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4d331-5e8b-4f80-83d6-c97b152db9b0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:54.214530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54807d-2cab-4d68-8b6e-0701e77313be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:54.215870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54807d-2cab-4d68-8b6e-0701e77313be None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.215870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54807d-2cab-4d68-8b6e-0701e77313be None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.215870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54807d-2cab-4d68-8b6e-0701e77313be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.231928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54807d-2cab-4d68-8b6e-0701e77313be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ef54807d-2cab-4d68-8b6e-0701e77313be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:54.238275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:54.239235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:54.240082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 185/373] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:54.240703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54807d-2cab-4d68-8b6e-0701e77313be None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:54.259913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593fc689-f622-405d-9bc5-355f8181aa6a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:54.299861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:54.300863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.301394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.303125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.304044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] RBAC: Adding query filter params (group.id=98e2773c36d4495da03871c9871b770b, role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=bb88575700f64da982e71dffc4bac6e9) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:54.304553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:54.306305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:54.312649 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 189/374] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:57:54 2022] GET /identity/v3/role_assignments?group.id=98e2773c36d4495da03871c9871b770b&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=5491e6c9891843b8979a054e900d2e34 => generated 260 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:54.313726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:54.375249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54807d-2cab-4d68-8b6e-0701e77313be None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:54.402090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:54.409528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:54.423552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2255ddda-98a7-4d1a-87cb-c552ae19817d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:54.424170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ddda-98a7-4d1a-87cb-c552ae19817d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.424444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ddda-98a7-4d1a-87cb-c552ae19817d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.424713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2255ddda-98a7-4d1a-87cb-c552ae19817d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.425446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ddda-98a7-4d1a-87cb-c552ae19817d None admin] RBAC: Adding query filter params (group.id=98e2773c36d4495da03871c9871b770b, role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=bb88575700f64da982e71dffc4bac6e9) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:54.425781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ddda-98a7-4d1a-87cb-c552ae19817d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:54.428781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2255ddda-98a7-4d1a-87cb-c552ae19817d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:54.437103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 186/375] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:57:54 2022] GET /identity/v3/role_assignments?group.id=98e2773c36d4495da03871c9871b770b&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=7b4a508ded224dfc812c05d174d41511 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:54.448188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e2d2e1-240a-40e1-9abc-f0f0a63055e6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:54.513294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3215f-8938-4301-bc55-1997beefddb7 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.513580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3215f-8938-4301-bc55-1997beefddb7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.513863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3215f-8938-4301-bc55-1997beefddb7 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.516626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 187/376] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:54.523492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:54.524195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.524473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.524741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.525421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:54.525744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:54.527169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:54.532831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0352-45ac-4021-9c44-4b501975561c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.533140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0352-45ac-4021-9c44-4b501975561c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.533382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0352-45ac-4021-9c44-4b501975561c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.538072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 190/377] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:54.544162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:54.575374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c0ae0352-45ac-4021-9c44-4b501975561c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:54.576650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c0ae0352-45ac-4021-9c44-4b501975561c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:54.693327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 188/378] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:54.711543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d05fbcc-ae0e-4770-b211-32a0315013fc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:54.751134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9619-8f88-4217-bb55-961ca1c75826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:54.751765 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9619-8f88-4217-bb55-961ca1c75826 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:54.752050 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9619-8f88-4217-bb55-961ca1c75826 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:54.752321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8f9619-8f88-4217-bb55-961ca1c75826 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:54.760634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:54.840412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f9619-8f88-4217-bb55-961ca1c75826 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:54.841851 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8f9619-8f88-4217-bb55-961ca1c75826 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:54.958998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 191/379] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:57:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 13:57:54.969595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae0352-45ac-4021-9c44-4b501975561c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:55.097044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae0352-45ac-4021-9c44-4b501975561c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:55.108331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:55.130211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9619-8f88-4217-bb55-961ca1c75826 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:55.143861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e9a64-e870-4d27-81ab-3a7cf3476584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:55.144522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e9a64-e870-4d27-81ab-3a7cf3476584 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.144824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e9a64-e870-4d27-81ab-3a7cf3476584 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.145431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e9a64-e870-4d27-81ab-3a7cf3476584 None admin] PATH_INFO: `/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.146656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e9a64-e870-4d27-81ab-3a7cf3476584 None admin] RBAC: Authorizing `identity:get_project(project_id=8abb8ea74a4d492daea0a4d1c426e1f2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:55.148101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e9a64-e870-4d27-81ab-3a7cf3476584 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:55.149819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 189/380] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:54 2022] GET /identity/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2 => generated 344 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:55.155431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e9a64-e870-4d27-81ab-3a7cf3476584 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:55.173592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:55.264717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8f9619-8f88-4217-bb55-961ca1c75826 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:55.282762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e9a64-e870-4d27-81ab-3a7cf3476584 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:55.309751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894edaa0-35e2-4054-9227-edaeba57642f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:55.310408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894edaa0-35e2-4054-9227-edaeba57642f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.310684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894edaa0-35e2-4054-9227-edaeba57642f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.310951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894edaa0-35e2-4054-9227-edaeba57642f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.329397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894edaa0-35e2-4054-9227-edaeba57642f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-894edaa0-35e2-4054-9227-edaeba57642f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:55.334391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:55.335287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:55.336090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 192/381] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:55.345763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec90d-a9fb-418c-8027-c609144c78aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:55.346419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec90d-a9fb-418c-8027-c609144c78aa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.346697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec90d-a9fb-418c-8027-c609144c78aa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.346971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec90d-a9fb-418c-8027-c609144c78aa None admin] PATH_INFO: `/v3/groups/7706ea63067c4e6b9d595c6b914262cf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.348448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894edaa0-35e2-4054-9227-edaeba57642f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:55.364048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ec90d-a9fb-418c-8027-c609144c78aa None admin] RBAC: Authorizing `identity:get_group(group_id=7706ea63067c4e6b9d595c6b914262cf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:55.365535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ec90d-a9fb-418c-8027-c609144c78aa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:55.367255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 190/382] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:55 2022] GET /identity/v3/groups/7706ea63067c4e6b9d595c6b914262cf => generated 229 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:55.372415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e75ce-61bb-437f-b70f-f13e27ced790 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.372699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e75ce-61bb-437f-b70f-f13e27ced790 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.372975 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1e75ce-61bb-437f-b70f-f13e27ced790 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.374284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 191/383] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:57:55.374974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e75ce-61bb-437f-b70f-f13e27ced790 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:55.433575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:55.474179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:55.484755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894edaa0-35e2-4054-9227-edaeba57642f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:55.523210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1e75ce-61bb-437f-b70f-f13e27ced790 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:55.541611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6afaf-ed5e-45d0-93d2-d3d2852f803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:55.542363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6afaf-ed5e-45d0-93d2-d3d2852f803f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.542644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6afaf-ed5e-45d0-93d2-d3d2852f803f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.543018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6afaf-ed5e-45d0-93d2-d3d2852f803f None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.543703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6afaf-ed5e-45d0-93d2-d3d2852f803f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:55.544008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6afaf-ed5e-45d0-93d2-d3d2852f803f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:55.545539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6afaf-ed5e-45d0-93d2-d3d2852f803f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:55.556703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 193/384] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:55.563169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579d38b-2fcd-4eeb-aa8b-b1e1b756e7cb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.563454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579d38b-2fcd-4eeb-aa8b-b1e1b756e7cb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.563759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579d38b-2fcd-4eeb-aa8b-b1e1b756e7cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.566509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:55.567288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.567564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.567836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.568850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] RBAC: Adding query filter params (group.id=7706ea63067c4e6b9d595c6b914262cf, role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=8abb8ea74a4d492daea0a4d1c426e1f2) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:55.569161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:55.570736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:55.578732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 192/385] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:57:55 2022] GET /identity/v3/role_assignments?group.id=7706ea63067c4e6b9d595c6b914262cf&scope.project.id=8abb8ea74a4d492daea0a4d1c426e1f2&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:55.580196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:55.588930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8579d38b-2fcd-4eeb-aa8b-b1e1b756e7cb None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:55.590217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8579d38b-2fcd-4eeb-aa8b-b1e1b756e7cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:55.687020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 194/386] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:55.775264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab2402e-0deb-44a9-8cbe-864984f3fb26 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:55.778216 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:55.814104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515589-f2eb-4d9b-b805-f9898d79bad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:55.815107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515589-f2eb-4d9b-b805-f9898d79bad4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.815624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515589-f2eb-4d9b-b805-f9898d79bad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.816098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3515589-f2eb-4d9b-b805-f9898d79bad4 None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.817551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3515589-f2eb-4d9b-b805-f9898d79bad4 None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:55.819306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3515589-f2eb-4d9b-b805-f9898d79bad4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:55.821079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 193/387] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:55 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:55.827597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579d38b-2fcd-4eeb-aa8b-b1e1b756e7cb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:55.938727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:55.952546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579d38b-2fcd-4eeb-aa8b-b1e1b756e7cb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:55.987282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55cbd5-1a82-4bad-9688-2dbe64dd23f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:55.988251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55cbd5-1a82-4bad-9688-2dbe64dd23f5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:55.988817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55cbd5-1a82-4bad-9688-2dbe64dd23f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:55.989282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55cbd5-1a82-4bad-9688-2dbe64dd23f5 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:55.990793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55cbd5-1a82-4bad-9688-2dbe64dd23f5 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:55.992417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55cbd5-1a82-4bad-9688-2dbe64dd23f5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:55.994618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 195/388] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:55 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:56.000492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515589-f2eb-4d9b-b805-f9898d79bad4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:56.064964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55cbd5-1a82-4bad-9688-2dbe64dd23f5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:56.188566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55cbd5-1a82-4bad-9688-2dbe64dd23f5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:56.191968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3515589-f2eb-4d9b-b805-f9898d79bad4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:56.227520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a506a-fc01-42ac-807e-99e30f604c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:56.228160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a506a-fc01-42ac-807e-99e30f604c6a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:56.228441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a506a-fc01-42ac-807e-99e30f604c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:56.228710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33a506a-fc01-42ac-807e-99e30f604c6a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:56.241934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33a506a-fc01-42ac-807e-99e30f604c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f33a506a-fc01-42ac-807e-99e30f604c6a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:56.255902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:56.256902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:56.257728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:56.264214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 196/389] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:56.268378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a506a-fc01-42ac-807e-99e30f604c6a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:56.284184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ba1e7-418e-4752-9d84-e0e239220082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:56.284864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ba1e7-418e-4752-9d84-e0e239220082 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:56.285139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ba1e7-418e-4752-9d84-e0e239220082 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:56.285461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ba1e7-418e-4752-9d84-e0e239220082 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5/groups/98e2773c36d4495da03871c9871b770b/roles/5491e6c9891843b8979a054e900d2e34` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:56.289558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ba1e7-418e-4752-9d84-e0e239220082 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9df1c54939a4373af874595595358c5, group_id=98e2773c36d4495da03871c9871b770b, role_id=5491e6c9891843b8979a054e900d2e34)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:56.291196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ba1e7-418e-4752-9d84-e0e239220082 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:56.300575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:56.314532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 194/390] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 13:57:55 2022] PUT /identity/v3/projects/d9df1c54939a4373af874595595358c5/groups/98e2773c36d4495da03871c9871b770b/roles/5491e6c9891843b8979a054e900d2e34 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:56.446814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33a506a-fc01-42ac-807e-99e30f604c6a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:56.510673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:56.511875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:56.511875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:56.512022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:56.512670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:56.512976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:56.514637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:56.529423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 197/391] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:56.538809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ba1e7-418e-4752-9d84-e0e239220082 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:56.610753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:56.697033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ba1e7-418e-4752-9d84-e0e239220082 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:56.738962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c220d-97f7-43a8-b0a6-0a231196095c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:56.739614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c220d-97f7-43a8-b0a6-0a231196095c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:56.739889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c220d-97f7-43a8-b0a6-0a231196095c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:56.740159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204c220d-97f7-43a8-b0a6-0a231196095c None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:56.742346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c220d-97f7-43a8-b0a6-0a231196095c None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:56.743801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204c220d-97f7-43a8-b0a6-0a231196095c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:56.745716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 195/392] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:56 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:56.751125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:56.850006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:56.885691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101aff4-4b5f-42f0-bb86-dc727b5dd765 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:56.934954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd1065-c34a-43d0-8c07-d77a0971e7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:56.935608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1065-c34a-43d0-8c07-d77a0971e7b0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:56.935885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1065-c34a-43d0-8c07-d77a0971e7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:56.936158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd1065-c34a-43d0-8c07-d77a0971e7b0 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:56.937311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd1065-c34a-43d0-8c07-d77a0971e7b0 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:56.938787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd1065-c34a-43d0-8c07-d77a0971e7b0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:56.940747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 198/393] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:56 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:56.946697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c220d-97f7-43a8-b0a6-0a231196095c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:57.069939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:57.072882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204c220d-97f7-43a8-b0a6-0a231196095c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:57.110646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e6c5e3-8ac7-426f-a24e-d6307281f6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:57.111325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6c5e3-8ac7-426f-a24e-d6307281f6dc None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:57.111926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6c5e3-8ac7-426f-a24e-d6307281f6dc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:57.112200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e6c5e3-8ac7-426f-a24e-d6307281f6dc None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5/groups/98e2773c36d4495da03871c9871b770b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:57.116726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6c5e3-8ac7-426f-a24e-d6307281f6dc None admin] RBAC: Authorizing `identity:create_grant(project_id=d9df1c54939a4373af874595595358c5, group_id=98e2773c36d4495da03871c9871b770b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:57.118205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e6c5e3-8ac7-426f-a24e-d6307281f6dc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:57.149813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 196/394] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 13:57:56 2022] PUT /identity/v3/projects/d9df1c54939a4373af874595595358c5/groups/98e2773c36d4495da03871c9871b770b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:57.273793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5536f43-5ca3-4b3f-a03f-b84ca0bacd6b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:57.274329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5536f43-5ca3-4b3f-a03f-b84ca0bacd6b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:57.274329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5536f43-5ca3-4b3f-a03f-b84ca0bacd6b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:57.275688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 199/395] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:57.282012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac293d2-ab41-4d99-9897-40b037664886 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:57.282895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac293d2-ab41-4d99-9897-40b037664886 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:57.283388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac293d2-ab41-4d99-9897-40b037664886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:57.310878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4ac293d2-ab41-4d99-9897-40b037664886 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:57.312462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4ac293d2-ab41-4d99-9897-40b037664886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:57.393289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 197/396] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:57.493425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1bcd-de6b-410c-9746-aeeae1ddf2fe None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:57.493792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1bcd-de6b-410c-9746-aeeae1ddf2fe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:57.494234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d1bcd-de6b-410c-9746-aeeae1ddf2fe None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:57.495751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 200/397] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:57.504707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450fdb-0fc5-471a-a065-029de1b6ee82 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:57.504707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450fdb-0fc5-471a-a065-029de1b6ee82 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:57.504707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f450fdb-0fc5-471a-a065-029de1b6ee82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:57.528748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1f450fdb-0fc5-471a-a065-029de1b6ee82 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:57.529955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f450fdb-0fc5-471a-a065-029de1b6ee82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:57.622396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 198/398] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:57.707867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d1bcd-de6b-410c-9746-aeeae1ddf2fe None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:57.839124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d1bcd-de6b-410c-9746-aeeae1ddf2fe None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:57.882588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141283dc-f56f-478b-99df-1de54e975d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:57.883228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141283dc-f56f-478b-99df-1de54e975d7e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:57.883502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141283dc-f56f-478b-99df-1de54e975d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:57.883769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141283dc-f56f-478b-99df-1de54e975d7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:57.900014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141283dc-f56f-478b-99df-1de54e975d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-141283dc-f56f-478b-99df-1de54e975d7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:57.904781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:57.905924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:57.906706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:57.910588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 201/399] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:57.916501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f450fdb-0fc5-471a-a065-029de1b6ee82 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:57.966567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:58.097296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f450fdb-0fc5-471a-a065-029de1b6ee82 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:58.148370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:58.148990 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:58.149269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:58.149539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:58.150236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:58.150538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:58.151902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:58.162714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 199/400] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:58.168180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141283dc-f56f-478b-99df-1de54e975d7e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:58.184485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:58.185893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:58.322811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:58.355852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e4cee4-7490-42af-8098-09ea51d4e460 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:58.391543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141283dc-f56f-478b-99df-1de54e975d7e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:58.406014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac25fbc-23c6-4276-8af8-3c78e55a4434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:58.406662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac25fbc-23c6-4276-8af8-3c78e55a4434 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:58.406937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac25fbc-23c6-4276-8af8-3c78e55a4434 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:58.407205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac25fbc-23c6-4276-8af8-3c78e55a4434 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:58.435109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac25fbc-23c6-4276-8af8-3c78e55a4434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6ac25fbc-23c6-4276-8af8-3c78e55a4434 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:58.450206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:58.451229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:58.451995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 200/401] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:57:58.459001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac25fbc-23c6-4276-8af8-3c78e55a4434 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:58.470203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dd4ed-b48c-4e93-a9d4-5de2c86f3b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:58.470855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dd4ed-b48c-4e93-a9d4-5de2c86f3b1f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:58.471132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dd4ed-b48c-4e93-a9d4-5de2c86f3b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:58.471403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44dd4ed-b48c-4e93-a9d4-5de2c86f3b1f None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:58.472525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dd4ed-b48c-4e93-a9d4-5de2c86f3b1f None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:58.474053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44dd4ed-b48c-4e93-a9d4-5de2c86f3b1f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:58.476921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 202/402] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:58 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:58.482586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dd4ed-b48c-4e93-a9d4-5de2c86f3b1f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:58.578112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:58.606356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44dd4ed-b48c-4e93-a9d4-5de2c86f3b1f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:58.640494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac25fbc-23c6-4276-8af8-3c78e55a4434 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:58.656662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae30748-1954-49c4-bc58-0bd556e1de3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:58.657647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae30748-1954-49c4-bc58-0bd556e1de3d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:58.658004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae30748-1954-49c4-bc58-0bd556e1de3d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:58.658595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae30748-1954-49c4-bc58-0bd556e1de3d None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:58.660092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae30748-1954-49c4-bc58-0bd556e1de3d None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:58.661840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae30748-1954-49c4-bc58-0bd556e1de3d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:58.664731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 203/403] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:58 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:57:58.666025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef889700-1aa5-4852-8d62-8bebdb5034b8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:58.666307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef889700-1aa5-4852-8d62-8bebdb5034b8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:58.666583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef889700-1aa5-4852-8d62-8bebdb5034b8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:58.667941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 204/404] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:58.673412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef889700-1aa5-4852-8d62-8bebdb5034b8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:58.685404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0937cb51-1818-43ca-98f4-b76e964e8467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:58.686187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937cb51-1818-43ca-98f4-b76e964e8467 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:58.686866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937cb51-1818-43ca-98f4-b76e964e8467 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:58.687218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0937cb51-1818-43ca-98f4-b76e964e8467 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:58.687982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937cb51-1818-43ca-98f4-b76e964e8467 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:58.688649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937cb51-1818-43ca-98f4-b76e964e8467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:58.690695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0937cb51-1818-43ca-98f4-b76e964e8467 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:58.704396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 201/405] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:57:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:58.705608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b3206-3cd9-462b-97ef-2b268d29019d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:58.705923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b3206-3cd9-462b-97ef-2b268d29019d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:58.706195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685b3206-3cd9-462b-97ef-2b268d29019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:58.717853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:58.733255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-685b3206-3cd9-462b-97ef-2b268d29019d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:58.734955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-685b3206-3cd9-462b-97ef-2b268d29019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:58.797274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef889700-1aa5-4852-8d62-8bebdb5034b8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:58.821540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 202/406] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:58.827391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b3206-3cd9-462b-97ef-2b268d29019d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:58.844293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59707c0b-99e6-4759-ab3d-bf024c864f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:58.845123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59707c0b-99e6-4759-ab3d-bf024c864f2d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:58.845517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59707c0b-99e6-4759-ab3d-bf024c864f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:58.845950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59707c0b-99e6-4759-ab3d-bf024c864f2d None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9/groups/98e2773c36d4495da03871c9871b770b/roles/5491e6c9891843b8979a054e900d2e34` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:58.848427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59707c0b-99e6-4759-ab3d-bf024c864f2d None admin] RBAC: Authorizing `identity:create_grant(project_id=bb88575700f64da982e71dffc4bac6e9, group_id=98e2773c36d4495da03871c9871b770b, role_id=5491e6c9891843b8979a054e900d2e34)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:58.857664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59707c0b-99e6-4759-ab3d-bf024c864f2d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:58.881777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 205/407] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 13:57:58 2022] PUT /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9/groups/98e2773c36d4495da03871c9871b770b/roles/5491e6c9891843b8979a054e900d2e34 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:58.996562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685b3206-3cd9-462b-97ef-2b268d29019d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:59.049054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb23568-e0db-4154-8e1b-28281d80319a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:59.049741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb23568-e0db-4154-8e1b-28281d80319a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.050022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb23568-e0db-4154-8e1b-28281d80319a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.050292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb23568-e0db-4154-8e1b-28281d80319a None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.051487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb23568-e0db-4154-8e1b-28281d80319a None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:59.052922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb23568-e0db-4154-8e1b-28281d80319a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:59.054526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 203/408] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:58 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:59.060611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59707c0b-99e6-4759-ab3d-bf024c864f2d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:59.063651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:59.168008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb23568-e0db-4154-8e1b-28281d80319a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:59.170984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:59.215806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59707c0b-99e6-4759-ab3d-bf024c864f2d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:59.270963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e19a47-9418-4720-97c0-94e3e7e2d11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:59.272028 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19a47-9418-4720-97c0-94e3e7e2d11d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.272314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19a47-9418-4720-97c0-94e3e7e2d11d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.273054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e19a47-9418-4720-97c0-94e3e7e2d11d None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.274344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19a47-9418-4720-97c0-94e3e7e2d11d None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:59.275797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e19a47-9418-4720-97c0-94e3e7e2d11d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:59.277620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 206/409] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:59 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:59.281047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215d0e9-eae2-4320-976f-3ccbd40da9d2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.281047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215d0e9-eae2-4320-976f-3ccbd40da9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.281047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c215d0e9-eae2-4320-976f-3ccbd40da9d2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.281799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 207/410] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:57:59.284402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215d0e9-eae2-4320-976f-3ccbd40da9d2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:59.311382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb23568-e0db-4154-8e1b-28281d80319a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:59.355488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af7f05-64e0-487f-ba77-b212c198fcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:59.357137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7f05-64e0-487f-ba77-b212c198fcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.357582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7f05-64e0-487f-ba77-b212c198fcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.358033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af7f05-64e0-487f-ba77-b212c198fcb3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.358662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:59.374992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af7f05-64e0-487f-ba77-b212c198fcb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-23af7f05-64e0-487f-ba77-b212c198fcb3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:59.381111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:59.382114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:57:59.382886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:57:59.384483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 204/411] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:57:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:59.386343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88549c13-eb4d-4fec-a1f4-6553b3907e51 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.386624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88549c13-eb4d-4fec-a1f4-6553b3907e51 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.386890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88549c13-eb4d-4fec-a1f4-6553b3907e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.401101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c215d0e9-eae2-4320-976f-3ccbd40da9d2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:59.405979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-88549c13-eb4d-4fec-a1f4-6553b3907e51 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:57:59.407173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-88549c13-eb4d-4fec-a1f4-6553b3907e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:57:59.440518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33098dd-a3b6-4e97-b37b-1021423e6f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:59.441687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33098dd-a3b6-4e97-b37b-1021423e6f9d None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.442182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33098dd-a3b6-4e97-b37b-1021423e6f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.442760 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33098dd-a3b6-4e97-b37b-1021423e6f9d None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9/groups/98e2773c36d4495da03871c9871b770b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.445035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33098dd-a3b6-4e97-b37b-1021423e6f9d None admin] RBAC: Authorizing `identity:create_grant(project_id=bb88575700f64da982e71dffc4bac6e9, group_id=98e2773c36d4495da03871c9871b770b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:59.446790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33098dd-a3b6-4e97-b37b-1021423e6f9d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:59.463750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 208/412] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 13:57:59 2022] PUT /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9/groups/98e2773c36d4495da03871c9871b770b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:57:59.465137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33098dd-a3b6-4e97-b37b-1021423e6f9d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:59.466542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:59.484179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 205/413] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:57:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:57:59.649196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33098dd-a3b6-4e97-b37b-1021423e6f9d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:59.688605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:57:59.696716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:59.697273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.697549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.697854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.698534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:57:59.698839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:59.700253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:59.709776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 209/414] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:57:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:59.715825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88549c13-eb4d-4fec-a1f4-6553b3907e51 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:59.791404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:59.846547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88549c13-eb4d-4fec-a1f4-6553b3907e51 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:59.892081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfefaa-99ae-4a34-8bb3-bf4880e436f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:59.893092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfefaa-99ae-4a34-8bb3-bf4880e436f2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.893559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfefaa-99ae-4a34-8bb3-bf4880e436f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.894094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cfefaa-99ae-4a34-8bb3-bf4880e436f2 None admin] PATH_INFO: `/v3/groups/7706ea63067c4e6b9d595c6b914262cf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.895615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfefaa-99ae-4a34-8bb3-bf4880e436f2 None admin] RBAC: Authorizing `identity:get_group(group_id=7706ea63067c4e6b9d595c6b914262cf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:57:59.897365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cfefaa-99ae-4a34-8bb3-bf4880e436f2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:57:59.899578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 206/415] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:57:59 2022] GET /identity/v3/groups/7706ea63067c4e6b9d595c6b914262cf => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:57:59.907513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfefaa-99ae-4a34-8bb3-bf4880e436f2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:57:59.916157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45df6c6-5edd-4a57-ad3a-4d3a0a1967c5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:57:59.971470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6deb6-5c4a-4a24-827d-005ff1b35e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:57:59.972034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6deb6-5c4a-4a24-827d-005ff1b35e69 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.972308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6deb6-5c4a-4a24-827d-005ff1b35e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.972570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6deb6-5c4a-4a24-827d-005ff1b35e69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.984058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6deb6-5c4a-4a24-827d-005ff1b35e69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-07b6deb6-5c4a-4a24-827d-005ff1b35e69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:57:59.990148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:57:59.991088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:57:59.991875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:57:59.996312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 210/416] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:57:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:57:59.997628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb080-32d4-4b7b-96ff-e683cf2fe2b5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:57:59.997968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb080-32d4-4b7b-96ff-e683cf2fe2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:57:59.998245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cb080-32d4-4b7b-96ff-e683cf2fe2b5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:57:59.999787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 211/417] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:00.000523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb080-32d4-4b7b-96ff-e683cf2fe2b5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:00.005564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:00.053747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cfefaa-99ae-4a34-8bb3-bf4880e436f2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:00.071640 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:00.096825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bdbf52-5e11-426a-88a1-bfda84e29b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:00.097926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdbf52-5e11-426a-88a1-bfda84e29b22 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:00.098425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdbf52-5e11-426a-88a1-bfda84e29b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:00.098896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bdbf52-5e11-426a-88a1-bfda84e29b22 None admin] PATH_INFO: `/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:00.100626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdbf52-5e11-426a-88a1-bfda84e29b22 None admin] RBAC: Authorizing `identity:get_project(project_id=8abb8ea74a4d492daea0a4d1c426e1f2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:00.103269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bdbf52-5e11-426a-88a1-bfda84e29b22 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:00.106624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 207/418] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:57:59 2022] GET /identity/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2 => generated 344 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:00.108401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109bb2b-c2ab-41ac-bc55-c0c3b99bd396 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:00.108937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109bb2b-c2ab-41ac-bc55-c0c3b99bd396 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:00.109509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5109bb2b-c2ab-41ac-bc55-c0c3b99bd396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:00.131394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5109bb2b-c2ab-41ac-bc55-c0c3b99bd396 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:00.132381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cb080-32d4-4b7b-96ff-e683cf2fe2b5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:00.132795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5109bb2b-c2ab-41ac-bc55-c0c3b99bd396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:00.177628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:00.178630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:00.179093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:00.179540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:00.180378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:00.180839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:00.182982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:00.194193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 212/419] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:57:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:00.195472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:00.213114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 208/420] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:00.219213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109bb2b-c2ab-41ac-bc55-c0c3b99bd396 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:00.317754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:00.380048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686af1fa-9094-4a8d-8bf9-ea7380648d61 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:00.396957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5109bb2b-c2ab-41ac-bc55-c0c3b99bd396 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:00.410634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:00.438185 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715082f4-5d57-4f18-82a1-4e1a1feea482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:00.438807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715082f4-5d57-4f18-82a1-4e1a1feea482 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:00.439082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715082f4-5d57-4f18-82a1-4e1a1feea482 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:00.439410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715082f4-5d57-4f18-82a1-4e1a1feea482 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:00.472613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218e6b1-489a-4ec5-b973-bc5578fdc7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:00.473340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e6b1-489a-4ec5-b973-bc5578fdc7ef None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:00.473613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e6b1-489a-4ec5-b973-bc5578fdc7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:00.473911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f218e6b1-489a-4ec5-b973-bc5578fdc7ef None admin] PATH_INFO: `/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2/groups/7706ea63067c4e6b9d595c6b914262cf/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:00.478554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218e6b1-489a-4ec5-b973-bc5578fdc7ef None admin] RBAC: Authorizing `identity:create_grant(project_id=8abb8ea74a4d492daea0a4d1c426e1f2, group_id=7706ea63067c4e6b9d595c6b914262cf, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:00.480057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f218e6b1-489a-4ec5-b973-bc5578fdc7ef None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:00.508996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 213/421] 10.222.0.23 () {58 vars in 1442 bytes} [Wed Oct 5 13:58:00 2022] PUT /identity/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2/groups/7706ea63067c4e6b9d595c6b914262cf/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:00.542535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715082f4-5d57-4f18-82a1-4e1a1feea482 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:00.544334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715082f4-5d57-4f18-82a1-4e1a1feea482 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:00.557122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218e6b1-489a-4ec5-b973-bc5578fdc7ef None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:00.620327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:00.671233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f218e6b1-489a-4ec5-b973-bc5578fdc7ef None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:00.672599 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 209/422] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:00.678994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715082f4-5d57-4f18-82a1-4e1a1feea482 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:00.720952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa350-9e9b-49d3-9f91-9cee4a047797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:00.721620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa350-9e9b-49d3-9f91-9cee4a047797 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:00.722306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa350-9e9b-49d3-9f91-9cee4a047797 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:00.722541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa350-9e9b-49d3-9f91-9cee4a047797 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:00.748012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa350-9e9b-49d3-9f91-9cee4a047797 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d54aa350-9e9b-49d3-9f91-9cee4a047797 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:00.759022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:00.759969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:58:00.760773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:00.765723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 214/423] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:00.768966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa350-9e9b-49d3-9f91-9cee4a047797 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:00.807934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:00.876948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa350-9e9b-49d3-9f91-9cee4a047797 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:00.878598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715082f4-5d57-4f18-82a1-4e1a1feea482 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:00.911966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:00.912605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:00.912878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:00.913197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:00.913931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] RBAC: Adding query filter params (name=member) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:00.914237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:00.915651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:00.924533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 215/424] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:58:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:00.931763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:00.949853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f54c1-d421-4dd4-8110-b06229fed518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:00.950502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f54c1-d421-4dd4-8110-b06229fed518 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:00.950779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f54c1-d421-4dd4-8110-b06229fed518 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:00.951052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f54c1-d421-4dd4-8110-b06229fed518 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:00.952290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f54c1-d421-4dd4-8110-b06229fed518 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:00.954244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38f54c1-d421-4dd4-8110-b06229fed518 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:00.956514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 210/425] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:58:00 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:00.962104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f54c1-d421-4dd4-8110-b06229fed518 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:01.027898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:01.066303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:01.073473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f8a5c6-1e2c-4050-8c1e-d737aef5a75a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:01.120304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a21ed-a9da-42dd-9b21-437983a4ccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:01.121283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a21ed-a9da-42dd-9b21-437983a4ccc9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:01.121813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a21ed-a9da-42dd-9b21-437983a4ccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:01.122296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a21ed-a9da-42dd-9b21-437983a4ccc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:01.175146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38f54c1-d421-4dd4-8110-b06229fed518 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:01.227627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa768dd-28d5-41db-9bf8-411330148f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:01.228350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa768dd-28d5-41db-9bf8-411330148f10 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:01.228642 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa768dd-28d5-41db-9bf8-411330148f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:01.228896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa768dd-28d5-41db-9bf8-411330148f10 None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:01.231479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa768dd-28d5-41db-9bf8-411330148f10 None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:01.232927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa768dd-28d5-41db-9bf8-411330148f10 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:01.234524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 211/426] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:58:00 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:01.241132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa768dd-28d5-41db-9bf8-411330148f10 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:01.270612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a21ed-a9da-42dd-9b21-437983a4ccc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:01.272099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071a21ed-a9da-42dd-9b21-437983a4ccc9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:01.378856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:01.387444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa768dd-28d5-41db-9bf8-411330148f10 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:01.431886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:01.433331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:01.434806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:01.435286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:01.436454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] RBAC: Adding query filter params (group.id=98e2773c36d4495da03871c9871b770b, role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=d9df1c54939a4373af874595595358c5) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:01.438485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:01.440168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:01.451691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 212/427] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:58:01 2022] GET /identity/v3/role_assignments?group.id=98e2773c36d4495da03871c9871b770b&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=5491e6c9891843b8979a054e900d2e34 => generated 611 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:01.455748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:01.463363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 216/428] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:01.472189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:01.631102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fa7260-c664-41a7-aaeb-0753645ff2f8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:01.684236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973831b3-cd55-43f1-be74-40d81ccea137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:01.684809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973831b3-cd55-43f1-be74-40d81ccea137 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:01.685082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973831b3-cd55-43f1-be74-40d81ccea137 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:01.685345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973831b3-cd55-43f1-be74-40d81ccea137 None admin] PATH_INFO: `/v3/projects/d9df1c54939a4373af874595595358c5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:01.686543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973831b3-cd55-43f1-be74-40d81ccea137 None admin] RBAC: Authorizing `identity:get_project(project_id=d9df1c54939a4373af874595595358c5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:01.688090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973831b3-cd55-43f1-be74-40d81ccea137 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:01.689763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 213/429] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:58:01 2022] GET /identity/v3/projects/d9df1c54939a4373af874595595358c5 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:01.697620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a21ed-a9da-42dd-9b21-437983a4ccc9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:01.733231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e751-5968-4eef-8409-679bf7b84b1f None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:01.733574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e751-5968-4eef-8409-679bf7b84b1f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:01.733825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17e751-5968-4eef-8409-679bf7b84b1f None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:01.735383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 214/430] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:01.741344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4724-865e-40d5-b288-9f8601a72b51 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:01.741769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4724-865e-40d5-b288-9f8601a72b51 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:01.742198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8a4724-865e-40d5-b288-9f8601a72b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:01.762725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fd8a4724-865e-40d5-b288-9f8601a72b51 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:01.764084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fd8a4724-865e-40d5-b288-9f8601a72b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:01.786696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:01.821161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a21ed-a9da-42dd-9b21-437983a4ccc9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:01.846144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 215/431] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:01.872330 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a435b24-ccc4-4772-ac27-ec39cd5de6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:01.873136 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a435b24-ccc4-4772-ac27-ec39cd5de6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:01.873409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a435b24-ccc4-4772-ac27-ec39cd5de6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:01.873715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a435b24-ccc4-4772-ac27-ec39cd5de6a5 None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:01.874819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a435b24-ccc4-4772-ac27-ec39cd5de6a5 None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:01.876330 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a435b24-ccc4-4772-ac27-ec39cd5de6a5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:01.878284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 217/432] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:58:01 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:01.884223 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a4724-865e-40d5-b288-9f8601a72b51 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:01.992983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8a4724-865e-40d5-b288-9f8601a72b51 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:02.029043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20a1d02-6247-4643-8026-8b56efaaba21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:02.030014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1d02-6247-4643-8026-8b56efaaba21 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.030509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1d02-6247-4643-8026-8b56efaaba21 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.030939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20a1d02-6247-4643-8026-8b56efaaba21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.031838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a1d02-6247-4643-8026-8b56efaaba21 None admin] RBAC: Adding query filter params (group.id=98e2773c36d4495da03871c9871b770b, role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=d9df1c54939a4373af874595595358c5) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:02.032322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a1d02-6247-4643-8026-8b56efaaba21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:02.033944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20a1d02-6247-4643-8026-8b56efaaba21 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:02.040737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 216/433] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:58:01 2022] GET /identity/v3/role_assignments?group.id=98e2773c36d4495da03871c9871b770b&scope.project.id=d9df1c54939a4373af874595595358c5&role.id=7b4a508ded224dfc812c05d174d41511 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:02.178495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a435b24-ccc4-4772-ac27-ec39cd5de6a5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:02.202572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433190dd-dd4c-4815-af72-256edfac9d16 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.203456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433190dd-dd4c-4815-af72-256edfac9d16 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.203802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433190dd-dd4c-4815-af72-256edfac9d16 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.205562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 217/434] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:58:02.212046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9869d40-282d-4968-9006-a4a5eb55f602 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.212591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9869d40-282d-4968-9006-a4a5eb55f602 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.213050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9869d40-282d-4968-9006-a4a5eb55f602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.235157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f9869d40-282d-4968-9006-a4a5eb55f602 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:02.237097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f9869d40-282d-4968-9006-a4a5eb55f602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:02.251561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:02.257890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:02.300874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a435b24-ccc4-4772-ac27-ec39cd5de6a5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:02.331834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 218/435] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:58:02.339273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75c2c4-fc43-4f05-a078-248a1b10c705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:02.340181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75c2c4-fc43-4f05-a078-248a1b10c705 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.340598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75c2c4-fc43-4f05-a078-248a1b10c705 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.341005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c75c2c4-fc43-4f05-a078-248a1b10c705 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.352207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c75c2c4-fc43-4f05-a078-248a1b10c705 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5c75c2c4-fc43-4f05-a078-248a1b10c705 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:02.357878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:02.358959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:58:02.359957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:02.362693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 218/436] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:02.368810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9869d40-282d-4968-9006-a4a5eb55f602 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:02.471821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9869d40-282d-4968-9006-a4a5eb55f602 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:02.511669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:02.512211 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.512485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.512754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.513455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:02.513764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:02.515144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:02.524115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 219/437] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:02.529873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75c2c4-fc43-4f05-a078-248a1b10c705 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:02.597986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:02.615470 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:02.643900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c75c2c4-fc43-4f05-a078-248a1b10c705 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:02.674438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:02.690901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5567b57f-9653-4841-a435-ab0793d14bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:02.691708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b57f-9653-4841-a435-ab0793d14bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.691992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b57f-9653-4841-a435-ab0793d14bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.692264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5567b57f-9653-4841-a435-ab0793d14bd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.748049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f06c190-0cb7-467e-bf6b-7320364415b8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:02.778872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567b57f-9653-4841-a435-ab0793d14bd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:02.780594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5567b57f-9653-4841-a435-ab0793d14bd0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:02.788814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77d589-6bf8-475e-9f2c-d7a9e030f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:02.789477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d589-6bf8-475e-9f2c-d7a9e030f76c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.789801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d589-6bf8-475e-9f2c-d7a9e030f76c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.790071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77d589-6bf8-475e-9f2c-d7a9e030f76c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.801329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77d589-6bf8-475e-9f2c-d7a9e030f76c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-cf77d589-6bf8-475e-9f2c-d7a9e030f76c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:02.807512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:02.808384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:02.809203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 220/438] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:02.814507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77d589-6bf8-475e-9f2c-d7a9e030f76c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:02.885267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:02.891106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 219/439] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:02.891106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79665-e3d3-4fbe-900a-6f827a0267e8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.891106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79665-e3d3-4fbe-900a-6f827a0267e8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.891106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e79665-e3d3-4fbe-900a-6f827a0267e8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.891106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 220/440] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:02.892969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e79665-e3d3-4fbe-900a-6f827a0267e8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:02.915879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77d589-6bf8-475e-9f2c-d7a9e030f76c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:02.918405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:02.949822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98790984-ab40-4437-836b-cfd1dc4b6ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:02.950454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98790984-ab40-4437-836b-cfd1dc4b6ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.950728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98790984-ab40-4437-836b-cfd1dc4b6ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.950994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98790984-ab40-4437-836b-cfd1dc4b6ee1 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.951663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98790984-ab40-4437-836b-cfd1dc4b6ee1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:02.951976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98790984-ab40-4437-836b-cfd1dc4b6ee1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:02.953343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98790984-ab40-4437-836b-cfd1dc4b6ee1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:02.961730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 221/441] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:58:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:02.962817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf376c-ddf3-4974-965d-74880cd2f16f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:02.963097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf376c-ddf3-4974-965d-74880cd2f16f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:02.963378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf376c-ddf3-4974-965d-74880cd2f16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:02.981124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0ccf376c-ddf3-4974-965d-74880cd2f16f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:02.982366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0ccf376c-ddf3-4974-965d-74880cd2f16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:02.999299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e79665-e3d3-4fbe-900a-6f827a0267e8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.032848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12393653-9104-4d19-a31e-b0f5f0474e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.033409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12393653-9104-4d19-a31e-b0f5f0474e08 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.033720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12393653-9104-4d19-a31e-b0f5f0474e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.033988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12393653-9104-4d19-a31e-b0f5f0474e08 None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.035089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12393653-9104-4d19-a31e-b0f5f0474e08 None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.036623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12393653-9104-4d19-a31e-b0f5f0474e08 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:03.038271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 221/442] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:58:02 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:03.043879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12393653-9104-4d19-a31e-b0f5f0474e08 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.046029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 222/443] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:03.046906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf376c-ddf3-4974-965d-74880cd2f16f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.109533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:03.139445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:03.147686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf376c-ddf3-4974-965d-74880cd2f16f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.150017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12393653-9104-4d19-a31e-b0f5f0474e08 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.183936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0188cd-1158-420d-983f-7e503f0e844e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.184142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d854013-1f40-449c-8cfd-95bd21d5931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.184572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0188cd-1158-420d-983f-7e503f0e844e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.184733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d854013-1f40-449c-8cfd-95bd21d5931f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.184886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0188cd-1158-420d-983f-7e503f0e844e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.184981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d854013-1f40-449c-8cfd-95bd21d5931f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.185121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0188cd-1158-420d-983f-7e503f0e844e None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.185270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d854013-1f40-449c-8cfd-95bd21d5931f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.186224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0188cd-1158-420d-983f-7e503f0e844e None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.187805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0188cd-1158-420d-983f-7e503f0e844e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:03.189440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 223/444] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:03.195222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0188cd-1158-420d-983f-7e503f0e844e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.254483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d854013-1f40-449c-8cfd-95bd21d5931f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.255957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d854013-1f40-449c-8cfd-95bd21d5931f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:03.296532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0188cd-1158-420d-983f-7e503f0e844e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.328447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:03.333996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.334604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.334876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.335143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.336095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] RBAC: Adding query filter params (group.id=98e2773c36d4495da03871c9871b770b, role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=bb88575700f64da982e71dffc4bac6e9) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:03.336402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.337858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:03.344027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 224/445] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/role_assignments?group.id=98e2773c36d4495da03871c9871b770b&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=5491e6c9891843b8979a054e900d2e34 => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:03.344953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.354250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 222/446] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:03.355066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:03.360319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d854013-1f40-449c-8cfd-95bd21d5931f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.470613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1698e369-2f20-4b8b-a5fe-4c93069396cf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.477719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d854013-1f40-449c-8cfd-95bd21d5931f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.510312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3210d37-6aa2-4bde-a9e4-56867f3c9f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.511138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3210d37-6aa2-4bde-a9e4-56867f3c9f1c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.511468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3210d37-6aa2-4bde-a9e4-56867f3c9f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.511733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3210d37-6aa2-4bde-a9e4-56867f3c9f1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.522994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3210d37-6aa2-4bde-a9e4-56867f3c9f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:03.523483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d2cba-6486-423d-8f7f-6f04c0ba542c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.524742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d2cba-6486-423d-8f7f-6f04c0ba542c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.524742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d2cba-6486-423d-8f7f-6f04c0ba542c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.524742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d2cba-6486-423d-8f7f-6f04c0ba542c None admin] PATH_INFO: `/v3/projects/bb88575700f64da982e71dffc4bac6e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.526262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d2cba-6486-423d-8f7f-6f04c0ba542c None admin] RBAC: Authorizing `identity:get_project(project_id=bb88575700f64da982e71dffc4bac6e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.527928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54d2cba-6486-423d-8f7f-6f04c0ba542c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d3210d37-6aa2-4bde-a9e4-56867f3c9f1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:03.528030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:03.528881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:03.529694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:03.530287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 223/447] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/projects/bb88575700f64da982e71dffc4bac6e9 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:03.533850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 225/448] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:03.534820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d2cba-6486-423d-8f7f-6f04c0ba542c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.535790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3210d37-6aa2-4bde-a9e4-56867f3c9f1c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.599155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:03.599338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:03.639984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d2cba-6486-423d-8f7f-6f04c0ba542c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.642887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3210d37-6aa2-4bde-a9e4-56867f3c9f1c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.671812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.672417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.672694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.672962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.673618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:03.673947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.675335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:03.676247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ad8be-184f-4ee2-8507-27315515b60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.676878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ad8be-184f-4ee2-8507-27315515b60c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.677152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ad8be-184f-4ee2-8507-27315515b60c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.677427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ad8be-184f-4ee2-8507-27315515b60c None admin] PATH_INFO: `/v3/groups/98e2773c36d4495da03871c9871b770b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.678570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ad8be-184f-4ee2-8507-27315515b60c None admin] RBAC: Authorizing `identity:get_group(group_id=98e2773c36d4495da03871c9871b770b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.679982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ad8be-184f-4ee2-8507-27315515b60c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:03.681544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 226/449] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/groups/98e2773c36d4495da03871c9871b770b => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:03.688397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 224/450] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:03.688546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ad8be-184f-4ee2-8507-27315515b60c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.689307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:03.783574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ad8be-184f-4ee2-8507-27315515b60c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.785951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb53403-83c0-492e-86dc-9f045eaf57a2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:03.806152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:03.807048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:03.817069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.817702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.817982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.818249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.818971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] RBAC: Adding query filter params (group.id=98e2773c36d4495da03871c9871b770b, role.id=7b4a508ded224dfc812c05d174d41511, scope.project.id=bb88575700f64da982e71dffc4bac6e9) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:03.819277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.819585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ea0dc-2809-4b24-add1-f9e20f99b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:03.820198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ea0dc-2809-4b24-add1-f9e20f99b489 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:03.820476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ea0dc-2809-4b24-add1-f9e20f99b489 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:03.820705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:03.820791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405ea0dc-2809-4b24-add1-f9e20f99b489 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:03.826353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 227/451] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/role_assignments?group.id=98e2773c36d4495da03871c9871b770b&scope.project.id=bb88575700f64da982e71dffc4bac6e9&role.id=7b4a508ded224dfc812c05d174d41511 => generated 611 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:03.895355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ea0dc-2809-4b24-add1-f9e20f99b489 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:03.896787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405ea0dc-2809-4b24-add1-f9e20f99b489 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:04.001500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 225/452] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 13:58:04.007903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:04.115871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42999c1a-feaa-486b-9723-8be008bb48c8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:04.156071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589afe39-4dae-4610-a5ff-eb861980bf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:04.156750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589afe39-4dae-4610-a5ff-eb861980bf13 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:04.157016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589afe39-4dae-4610-a5ff-eb861980bf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:04.157279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589afe39-4dae-4610-a5ff-eb861980bf13 None admin] PATH_INFO: `/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:04.158511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589afe39-4dae-4610-a5ff-eb861980bf13 None admin] RBAC: Authorizing `identity:get_project(project_id=8abb8ea74a4d492daea0a4d1c426e1f2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:04.160004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589afe39-4dae-4610-a5ff-eb861980bf13 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:04.161911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 228/453] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:58:04 2022] GET /identity/v3/projects/8abb8ea74a4d492daea0a4d1c426e1f2 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:04.167589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ea0dc-2809-4b24-add1-f9e20f99b489 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:04.193011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:04.275958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405ea0dc-2809-4b24-add1-f9e20f99b489 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:04.310317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfd2f39-d84d-4f25-89cf-08be776627ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:04.310963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2f39-d84d-4f25-89cf-08be776627ab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:04.311243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2f39-d84d-4f25-89cf-08be776627ab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:04.311518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfd2f39-d84d-4f25-89cf-08be776627ab None admin] PATH_INFO: `/v3/groups/7706ea63067c4e6b9d595c6b914262cf` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:04.312687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd2f39-d84d-4f25-89cf-08be776627ab None admin] RBAC: Authorizing `identity:get_group(group_id=7706ea63067c4e6b9d595c6b914262cf)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:04.314178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfd2f39-d84d-4f25-89cf-08be776627ab None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:04.315758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 226/454] 10.222.0.23 () {56 vars in 1180 bytes} [Wed Oct 5 13:58:04 2022] GET /identity/v3/groups/7706ea63067c4e6b9d595c6b914262cf => generated 229 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:04.321802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589afe39-4dae-4610-a5ff-eb861980bf13 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:04.386459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:04.436932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589afe39-4dae-4610-a5ff-eb861980bf13 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:04.474115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89217b2-80fd-4a62-8e96-cbb86df76c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:04.474708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89217b2-80fd-4a62-8e96-cbb86df76c1e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:04.474981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89217b2-80fd-4a62-8e96-cbb86df76c1e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:04.475247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89217b2-80fd-4a62-8e96-cbb86df76c1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:04.475971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89217b2-80fd-4a62-8e96-cbb86df76c1e None admin] RBAC: Adding query filter params (group.id=7706ea63067c4e6b9d595c6b914262cf, role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=8abb8ea74a4d492daea0a4d1c426e1f2) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:04.476278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89217b2-80fd-4a62-8e96-cbb86df76c1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:04.477757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89217b2-80fd-4a62-8e96-cbb86df76c1e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:04.484268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 229/455] 10.222.0.23 () {56 vars in 1376 bytes} [Wed Oct 5 13:58:04 2022] GET /identity/v3/role_assignments?group.id=7706ea63067c4e6b9d595c6b914262cf&scope.project.id=8abb8ea74a4d492daea0a4d1c426e1f2&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 611 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:07.107413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224465ac-2dba-4f5d-8fda-031637d0506f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:07.107941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224465ac-2dba-4f5d-8fda-031637d0506f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:07.107941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224465ac-2dba-4f5d-8fda-031637d0506f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:07.109446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 227/456] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:07.116158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01fa1d-7a22-4145-9c8a-be5163ee863b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:07.116569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01fa1d-7a22-4145-9c8a-be5163ee863b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:07.116740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca01fa1d-7a22-4145-9c8a-be5163ee863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:07.147253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ca01fa1d-7a22-4145-9c8a-be5163ee863b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:07.148465 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca01fa1d-7a22-4145-9c8a-be5163ee863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:07.244279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 230/457] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:07.465025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70392d98-ee04-4edf-a0e1-76afc713fa38 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:07.465545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70392d98-ee04-4edf-a0e1-76afc713fa38 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:07.465984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70392d98-ee04-4edf-a0e1-76afc713fa38 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:07.467502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 228/458] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:07.472943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5434a9-78f7-4d96-80e3-c2d0111b548c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:07.473382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5434a9-78f7-4d96-80e3-c2d0111b548c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:07.473808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5434a9-78f7-4d96-80e3-c2d0111b548c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:07.501790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b4f96-460d-46e2-b672-33e8b1dcf414 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:07.502325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b4f96-460d-46e2-b672-33e8b1dcf414 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:07.502733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7b4f96-460d-46e2-b672-33e8b1dcf414 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:07.504251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 229/459] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:58:07.505148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bd5434a9-78f7-4d96-80e3-c2d0111b548c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:07.506578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bd5434a9-78f7-4d96-80e3-c2d0111b548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:07.509452 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b66c7-d903-4582-b7a3-0346bd9d1229 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:07.509905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b66c7-d903-4582-b7a3-0346bd9d1229 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:07.510304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16b66c7-d903-4582-b7a3-0346bd9d1229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:07.536743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f16b66c7-d903-4582-b7a3-0346bd9d1229 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:07.539374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f16b66c7-d903-4582-b7a3-0346bd9d1229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:07.605016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:07.646135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 231/460] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:07.647010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5434a9-78f7-4d96-80e3-c2d0111b548c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:07.675389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 230/461] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:58:07.785747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5434a9-78f7-4d96-80e3-c2d0111b548c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:07.828685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:07.829668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:07.830118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:07.830550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:07.847689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:07.849410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:07.864800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:07.869930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:07.870786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:07.870786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:07.870786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:07.872572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 232/462] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:07.878420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b66c7-d903-4582-b7a3-0346bd9d1229 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:07.905360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:07.945824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:08.027661 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16b66c7-d903-4582-b7a3-0346bd9d1229 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:08.028293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:08.068156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15474537-051f-45c8-aee5-30a673d76076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:08.069130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15474537-051f-45c8-aee5-30a673d76076 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.069573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15474537-051f-45c8-aee5-30a673d76076 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.070047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15474537-051f-45c8-aee5-30a673d76076 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.070835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15474537-051f-45c8-aee5-30a673d76076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:08.071281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15474537-051f-45c8-aee5-30a673d76076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:08.072952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15474537-051f-45c8-aee5-30a673d76076 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:08.086185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 231/463] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:08.087743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd01e3a-3c45-4209-afd8-af21470f08cc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.088162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd01e3a-3c45-4209-afd8-af21470f08cc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.088656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd01e3a-3c45-4209-afd8-af21470f08cc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.090170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 232/464] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:08.095810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd01e3a-3c45-4209-afd8-af21470f08cc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:08.125093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8afd301-4640-4b8b-81e8-2dbb1c99801a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:08.159784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cabe70-cabb-4953-b725-a238126bec53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:08.160909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cabe70-cabb-4953-b725-a238126bec53 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.161396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cabe70-cabb-4953-b725-a238126bec53 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.161828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cabe70-cabb-4953-b725-a238126bec53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.174462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cabe70-cabb-4953-b725-a238126bec53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:08.176329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cabe70-cabb-4953-b725-a238126bec53 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:08.186304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-91cabe70-cabb-4953-b725-a238126bec53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:08.192418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:08.193413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:08.194540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:08.194540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:08.194540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:08.196195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 233/465] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:08.197609 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68a92-5728-45e5-9dcb-e99f81301d51 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.198068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68a92-5728-45e5-9dcb-e99f81301d51 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.198490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec68a92-5728-45e5-9dcb-e99f81301d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.215651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd01e3a-3c45-4209-afd8-af21470f08cc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:08.216990 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-aec68a92-5728-45e5-9dcb-e99f81301d51 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:08.218212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aec68a92-5728-45e5-9dcb-e99f81301d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:08.257769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba4186-5cbe-4657-b55a-4222740cf7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:08.258402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba4186-5cbe-4657-b55a-4222740cf7d1 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.258677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba4186-5cbe-4657-b55a-4222740cf7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.258939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba4186-5cbe-4657-b55a-4222740cf7d1 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.259581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba4186-5cbe-4657-b55a-4222740cf7d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:08.261722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ba4186-5cbe-4657-b55a-4222740cf7d1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:08.297683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 234/466] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:08.297683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec68a92-5728-45e5-9dcb-e99f81301d51 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:08.297960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 233/467] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:58:08 2022] POST /identity/v3/users => generated 246 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:08.297960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba4186-5cbe-4657-b55a-4222740cf7d1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:08.337662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:08.463276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec68a92-5728-45e5-9dcb-e99f81301d51 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:08.470986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba4186-5cbe-4657-b55a-4222740cf7d1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:08.502760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099b26e5-2ea4-48d5-b892-e9a579ce1360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:08.503408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b26e5-2ea4-48d5-b892-e9a579ce1360 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.503688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b26e5-2ea4-48d5-b892-e9a579ce1360 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.504016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099b26e5-2ea4-48d5-b892-e9a579ce1360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.515825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b26e5-2ea4-48d5-b892-e9a579ce1360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:08.517595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099b26e5-2ea4-48d5-b892-e9a579ce1360 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:08.524230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b917db-94cc-44f2-a952-9b7fa3e04572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:08.524865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b917db-94cc-44f2-a952-9b7fa3e04572 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.525143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b917db-94cc-44f2-a952-9b7fa3e04572 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.525692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b917db-94cc-44f2-a952-9b7fa3e04572 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.526166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b917db-94cc-44f2-a952-9b7fa3e04572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:08.526468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b917db-94cc-44f2-a952-9b7fa3e04572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:08.528194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b917db-94cc-44f2-a952-9b7fa3e04572 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:08.530869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-099b26e5-2ea4-48d5-b892-e9a579ce1360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:08.537009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:08.537941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:08.538757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:08.538757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:08.538757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:08.538757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 235/468] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:08.539332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a8bb1-2583-4e2f-acc1-3e7549a5010b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.539605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a8bb1-2583-4e2f-acc1-3e7549a5010b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.539868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a8bb1-2583-4e2f-acc1-3e7549a5010b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.541166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 236/469] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:08.545567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a8bb1-2583-4e2f-acc1-3e7549a5010b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:08.549797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 234/470] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:08.551395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb656-c5a7-40af-b10e-96e9e21473e5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.551676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb656-c5a7-40af-b10e-96e9e21473e5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.551946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9bb656-c5a7-40af-b10e-96e9e21473e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.573559 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9a9bb656-c5a7-40af-b10e-96e9e21473e5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:08.574797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a9bb656-c5a7-40af-b10e-96e9e21473e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:08.624711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:08.654284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 235/471] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:08.655209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bb656-c5a7-40af-b10e-96e9e21473e5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:08.670662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949a8bb1-2583-4e2f-acc1-3e7549a5010b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:08.711068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:08.712400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.713117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.713578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.714540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:08.715010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:08.716628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:08.732010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 237/472] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:08.733286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:08.790774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9bb656-c5a7-40af-b10e-96e9e21473e5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:08.826527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:08.840021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b019e5-7a3c-47e1-829d-4560f2a2355b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:08.841148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b019e5-7a3c-47e1-829d-4560f2a2355b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.841514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b019e5-7a3c-47e1-829d-4560f2a2355b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.842146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b019e5-7a3c-47e1-829d-4560f2a2355b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.842897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b019e5-7a3c-47e1-829d-4560f2a2355b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:08.845358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b019e5-7a3c-47e1-829d-4560f2a2355b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:08.847216 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1a164-903b-4d0a-892a-5140b89c65b6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:08.873603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 236/473] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:58:08 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:08.876380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b019e5-7a3c-47e1-829d-4560f2a2355b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:08.885166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:08.900884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:08.901857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:08.902310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:08.902771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:08.916259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:08.918454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:08.936567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:08.937543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:08.938708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:08.938708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:08.938708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:08.940645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 238/474] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:08.941808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:09.038375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b019e5-7a3c-47e1-829d-4560f2a2355b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:09.065697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8addffb4-e013-4db1-9aa8-a0eed8c63bd2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:09.086058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00819035-1429-482f-abc3-e2abd02d3738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:09.087052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819035-1429-482f-abc3-e2abd02d3738 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:09.088462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819035-1429-482f-abc3-e2abd02d3738 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:09.088790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00819035-1429-482f-abc3-e2abd02d3738 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:09.089448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00819035-1429-482f-abc3-e2abd02d3738 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:09.091444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00819035-1429-482f-abc3-e2abd02d3738 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:09.108895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:09.109415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:09.109758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:09.110026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:09.110705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:09.112423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:09.120503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 237/475] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:58:08 2022] POST /identity/v3/users => generated 248 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:09.121536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00819035-1429-482f-abc3-e2abd02d3738 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:09.122116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.sql.core [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 05 13:58:09.122116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 05 13:58:09.122116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [parameters: {'id': '8ddeeec8df304e63bd41fa412995f9df', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 05 13:58:09.122116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81571) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Oct 05 13:58:09.138418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return ref.to_dict() Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 13:58:09.139766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application yield resource Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.rollback() Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.force_reraise() Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise self.value Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application session.commit() Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 05 13:58:09.140794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._prepare_impl() Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.flush() Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._flush(objects) Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application compat.raise_( Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application flush_context.execute() Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rec.execute(self) Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 05 13:58:09.141718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 13:58:09.150820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 13:58:09.152894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [parameters: {'id': '8ddeeec8df304e63bd41fa412995f9df', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:09.154908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 05 13:58:09.156883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:09.160084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 239/476] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 13:58:08 2022] POST /identity/v3/roles => generated 144 bytes in 198 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:09.160084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:09.213455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:09.225776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:09.295097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9632ef-0d58-4f72-85fd-b1ac0c8f0bd6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:09.318066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00819035-1429-482f-abc3-e2abd02d3738 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:09.332033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610dade-b58f-427c-87ec-08e41a37bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:09.332729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610dade-b58f-427c-87ec-08e41a37bf9f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:09.333001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610dade-b58f-427c-87ec-08e41a37bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:09.333318 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610dade-b58f-427c-87ec-08e41a37bf9f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:09.345712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2610dade-b58f-427c-87ec-08e41a37bf9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2610dade-b58f-427c-87ec-08e41a37bf9f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:09.351690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:09.352621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:09.353446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 240/477] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:09.370832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610dade-b58f-427c-87ec-08e41a37bf9f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:09.402309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:09.403783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:09.404064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:09.404334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:09.405073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:09.405382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:09.406958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:09.422998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 238/478] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:09.430651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:09.523840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610dade-b58f-427c-87ec-08e41a37bf9f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:09.526570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:09.557471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c7dc0-cbb4-43d4-95e7-b5d9f6901141 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:09.559574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:09.575632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036c921-d1b8-424a-9a1a-f054c0b77859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:09.576830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c921-d1b8-424a-9a1a-f054c0b77859 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:09.577739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c921-d1b8-424a-9a1a-f054c0b77859 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:09.578351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c921-d1b8-424a-9a1a-f054c0b77859 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:09.579351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036c921-d1b8-424a-9a1a-f054c0b77859 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:09.580019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036c921-d1b8-424a-9a1a-f054c0b77859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:09.582601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036c921-d1b8-424a-9a1a-f054c0b77859 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:09.594824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 241/479] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:09.600699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f27c2b-f42c-4fa4-b2d2-c76a86d6fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:09.601439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f27c2b-f42c-4fa4-b2d2-c76a86d6fd22 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:09.601815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f27c2b-f42c-4fa4-b2d2-c76a86d6fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:09.602156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f27c2b-f42c-4fa4-b2d2-c76a86d6fd22 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:09.602887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f27c2b-f42c-4fa4-b2d2-c76a86d6fd22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:09.604444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f27c2b-f42c-4fa4-b2d2-c76a86d6fd22 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:09.633217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 239/480] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:58:09 2022] POST /identity/v3/users => generated 249 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:10.821981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13a62-f77b-4fc5-8e6e-cd75f9b54421 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:10.822485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13a62-f77b-4fc5-8e6e-cd75f9b54421 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:10.822485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d13a62-f77b-4fc5-8e6e-cd75f9b54421 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:10.824079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 242/481] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:10.829375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa100e4-eadb-4664-b567-1033635319c6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:10.829829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa100e4-eadb-4664-b567-1033635319c6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:10.830001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa100e4-eadb-4664-b567-1033635319c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:10.858107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bfa100e4-eadb-4664-b567-1033635319c6 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:10.859352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bfa100e4-eadb-4664-b567-1033635319c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:10.946107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 240/482] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:11.251123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d13a62-f77b-4fc5-8e6e-cd75f9b54421 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:11.375968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d13a62-f77b-4fc5-8e6e-cd75f9b54421 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:11.418420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc77c-b6df-4867-abaa-9a55e6a6f88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:11.419055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc77c-b6df-4867-abaa-9a55e6a6f88e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:11.419325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc77c-b6df-4867-abaa-9a55e6a6f88e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:11.419589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fc77c-b6df-4867-abaa-9a55e6a6f88e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:11.440494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fc77c-b6df-4867-abaa-9a55e6a6f88e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8f6fc77c-b6df-4867-abaa-9a55e6a6f88e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:11.445337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:11.446252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:11.447121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 243/483] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:11.453990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa100e4-eadb-4664-b567-1033635319c6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:11.581361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:11.600585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa100e4-eadb-4664-b567-1033635319c6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:11.646168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd63eed-5d78-4124-9423-6751556ac48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:11.646791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd63eed-5d78-4124-9423-6751556ac48a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:11.647071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd63eed-5d78-4124-9423-6751556ac48a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:11.647342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd63eed-5d78-4124-9423-6751556ac48a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:11.648016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd63eed-5d78-4124-9423-6751556ac48a None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:11.648321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd63eed-5d78-4124-9423-6751556ac48a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:11.649734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd63eed-5d78-4124-9423-6751556ac48a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:11.658439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 241/484] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:11.664217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc77c-b6df-4867-abaa-9a55e6a6f88e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:11.717021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:11.787207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e53c3-16d6-40b5-a97e-6daa421f955a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:11.787375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e53c3-16d6-40b5-a97e-6daa421f955a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:11.787636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e53c3-16d6-40b5-a97e-6daa421f955a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:11.789312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 242/485] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:58:11.798902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148f3d4-fd34-4d0d-a1af-ef6d40674cc0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:11.799185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148f3d4-fd34-4d0d-a1af-ef6d40674cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:11.799461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a148f3d4-fd34-4d0d-a1af-ef6d40674cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:11.812555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fc77c-b6df-4867-abaa-9a55e6a6f88e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:11.835998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a148f3d4-fd34-4d0d-a1af-ef6d40674cc0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:11.838073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a148f3d4-fd34-4d0d-a1af-ef6d40674cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:11.853835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:11.854504 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:11.854778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:11.855046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:11.867753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:11.869259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:11.885605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:11.886552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:11.887372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:11.887372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:11.887372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:11.887372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 244/486] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:11.893213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:11.986790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 243/487] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:12.033783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:12.061748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3829240-4982-44e2-8e3f-ecd145acf9e2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:12.101353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6888a1f-844e-4a22-aa31-6109dd371e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:12.101996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6888a1f-844e-4a22-aa31-6109dd371e98 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.102265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6888a1f-844e-4a22-aa31-6109dd371e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.102527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6888a1f-844e-4a22-aa31-6109dd371e98 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.103165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6888a1f-844e-4a22-aa31-6109dd371e98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:12.103456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6888a1f-844e-4a22-aa31-6109dd371e98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:12.104815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6888a1f-844e-4a22-aa31-6109dd371e98 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:12.121177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 245/488] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:12.127053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148f3d4-fd34-4d0d-a1af-ef6d40674cc0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:12.160283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba777e-71bb-4203-98c4-3357de2585fa None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.160608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba777e-71bb-4203-98c4-3357de2585fa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.160817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba777e-71bb-4203-98c4-3357de2585fa None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.162345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 246/489] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:58:12.171238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d30009-a0a7-49c8-a3a1-5d79f7869873 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.171508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d30009-a0a7-49c8-a3a1-5d79f7869873 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.171774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d30009-a0a7-49c8-a3a1-5d79f7869873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.204835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-34d30009-a0a7-49c8-a3a1-5d79f7869873 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:12.206066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34d30009-a0a7-49c8-a3a1-5d79f7869873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:12.254026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:12.297352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a148f3d4-fd34-4d0d-a1af-ef6d40674cc0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:12.318128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 247/490] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:58:12.318912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d30009-a0a7-49c8-a3a1-5d79f7869873 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:12.343205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:12.343838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.344111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.344374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.352051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:12.353500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:12.368187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:12.369565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:12.370494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 244/491] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:58:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:12.377891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:12.403471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:12.470968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d30009-a0a7-49c8-a3a1-5d79f7869873 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:12.539737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51146243-24d1-428c-a057-512774be741e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:12.540686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51146243-24d1-428c-a057-512774be741e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.540968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51146243-24d1-428c-a057-512774be741e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.541239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51146243-24d1-428c-a057-512774be741e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.542065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766e2ab-3687-48bc-887d-fb22520f5b0a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:12.552302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51146243-24d1-428c-a057-512774be741e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-51146243-24d1-428c-a057-512774be741e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:12.557446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:12.558371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:12.559510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 248/492] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:12.560471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a74462-da23-46c1-8fba-b5bb7cbe590a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.560471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a74462-da23-46c1-8fba-b5bb7cbe590a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.560586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a74462-da23-46c1-8fba-b5bb7cbe590a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.565401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 249/493] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:12.565401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f2949-6654-46ce-a22f-e84b1fb03f40 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.565401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f2949-6654-46ce-a22f-e84b1fb03f40 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.565401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07f2949-6654-46ce-a22f-e84b1fb03f40 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.565401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 250/494] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:12.570014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f2949-6654-46ce-a22f-e84b1fb03f40 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:12.586164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897a739b-159a-4d25-b916-d087bf3435bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:12.586690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a739b-159a-4d25-b916-d087bf3435bc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.586961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a739b-159a-4d25-b916-d087bf3435bc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.587227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897a739b-159a-4d25-b916-d087bf3435bc None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.587962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a739b-159a-4d25-b916-d087bf3435bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:12.588264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a739b-159a-4d25-b916-d087bf3435bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:12.590061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897a739b-159a-4d25-b916-d087bf3435bc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:12.601345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:12.626645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 245/495] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:58:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:12.628501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926779e-985d-4a82-b782-446222248771 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.629153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926779e-985d-4a82-b782-446222248771 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.629514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8926779e-985d-4a82-b782-446222248771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.648077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8926779e-985d-4a82-b782-446222248771 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:12.649268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8926779e-985d-4a82-b782-446222248771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:12.677405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07f2949-6654-46ce-a22f-e84b1fb03f40 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:12.702642 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:12.712172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:12.712784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.713062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.722196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 246/496] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:12.722315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.722315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:12.722315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:12.722643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d03c2-d08e-4d60-adb4-fff04c19f81f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.722643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d03c2-d08e-4d60-adb4-fff04c19f81f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.722643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d03c2-d08e-4d60-adb4-fff04c19f81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.722811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:12.729570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 251/497] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:12.729570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:12.737292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ef0d03c2-d08e-4d60-adb4-fff04c19f81f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:12.738585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ef0d03c2-d08e-4d60-adb4-fff04c19f81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:12.811331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 247/498] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:12.818757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d03c2-d08e-4d60-adb4-fff04c19f81f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:12.844532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3976a365-0ad5-4f67-bfe8-8b19017364fb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:12.876246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:12.889722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cd0ef-0b71-4712-93e0-df0fed22dfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:12.891241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cd0ef-0b71-4712-93e0-df0fed22dfac None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.891766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cd0ef-0b71-4712-93e0-df0fed22dfac None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.892234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31cd0ef-0b71-4712-93e0-df0fed22dfac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.921335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d03c2-d08e-4d60-adb4-fff04c19f81f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:12.955644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14041a-226d-4b42-9f14-db2be168b6b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:12.956281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14041a-226d-4b42-9f14-db2be168b6b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:12.956554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14041a-226d-4b42-9f14-db2be168b6b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:12.956846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab14041a-226d-4b42-9f14-db2be168b6b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:12.958386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:12.967490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab14041a-226d-4b42-9f14-db2be168b6b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ab14041a-226d-4b42-9f14-db2be168b6b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:12.972628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:12.973504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:12.974352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:12.975897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 248/499] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:12.976715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14041a-226d-4b42-9f14-db2be168b6b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:12.978746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cd0ef-0b71-4712-93e0-df0fed22dfac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:12.980207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31cd0ef-0b71-4712-93e0-df0fed22dfac None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.079308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab14041a-226d-4b42-9f14-db2be168b6b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.083048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 252/500] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:13.083885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cd0ef-0b71-4712-93e0-df0fed22dfac None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.094337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:13.114378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.114947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.115218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.115482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.125415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.126907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:13.140972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:13.141987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:13.143184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:13.143184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:13.143184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:13.143184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 249/501] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:13.143513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.173180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:13.188680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31cd0ef-0b71-4712-93e0-df0fed22dfac None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.224262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.225016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.225365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.225623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.238214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.240472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.247886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261f8d86-9777-4e5d-b546-0c5159b28461 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:13.258012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:13.258920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:13.259769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:13.259769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:13.259769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:13.259769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 253/502] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:13.260875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.281730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.282336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.282606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.282871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.283526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:13.283820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.285217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.293195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 250/503] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:13.294070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.324765 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:13.356749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb5b18-ddb9-4f10-8739-afd93c8a555a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.389370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48117be-0115-42e7-b83e-615fe26cd048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.389956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48117be-0115-42e7-b83e-615fe26cd048 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.390267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48117be-0115-42e7-b83e-615fe26cd048 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.390502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48117be-0115-42e7-b83e-615fe26cd048 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.396485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce0eb7b-181c-46be-a5b4-1ea23f7139b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.398113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:13.401575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48117be-0115-42e7-b83e-615fe26cd048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d48117be-0115-42e7-b83e-615fe26cd048 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:13.406785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:13.407650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:13.408525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:13.413051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 254/504] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:13.413926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48117be-0115-42e7-b83e-615fe26cd048 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.429838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.430503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.430778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.431045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.441562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.443133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:13.456621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:13.457669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:13.458586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:13.458586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:13.458586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:13.458586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 251/505] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:13.458832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.522267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48117be-0115-42e7-b83e-615fe26cd048 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.538496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:13.560576 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9d5fe-19ae-4156-bab8-6b9d6ed05c9e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.560819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.561249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.561511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.561801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.562422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:13.562733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.564089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.576113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 255/506] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:13.576917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.598727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.599342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.599618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.599888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.600530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:13.600837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.602247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.613741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 252/507] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:13.614522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.636068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:13.676284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cb4dbb-39bf-435c-af8b-8fdcdbdea25a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.710202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c52d19b-4223-4b91-932d-f097a643e88c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.710727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.711350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.711625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.711890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.721800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.723911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:13.738711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:13.739645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:13.741113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:13.741113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:13.741113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:13.741113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 256/508] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:13.741113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.743654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.744774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.744774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.744774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.745455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:13.745790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.747200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.755342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 253/509] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:13.756342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.758782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:13.851580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2e0c8-4c7e-42ef-a9d6-c6f02e8ed256 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.867305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86c2929-ac8b-4d62-9e82-af3d8f6c01e5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:13.875308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:13.899919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.900739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.901017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.901362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.902222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:13.902522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.903935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.908344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:13.909004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:13.909280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:13.909550 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:13.916319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 257/510] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:13.917652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:13.919139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:13.921754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:13.932165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:13.933065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:13.934001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 254/511] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:13.934546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.001230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:14.031165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a9dc40-c878-497d-a8b6-2ee509e93e7f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.044397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f6d68-a650-4557-a36a-c6dcb23e8569 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.069936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428975bb-b106-495f-9a3e-e53919318eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.070552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428975bb-b106-495f-9a3e-e53919318eb9 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.070826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428975bb-b106-495f-9a3e-e53919318eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.071099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428975bb-b106-495f-9a3e-e53919318eb9 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.071748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428975bb-b106-495f-9a3e-e53919318eb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.073188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428975bb-b106-495f-9a3e-e53919318eb9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.079177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.079788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.080064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.080335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.080995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:14.081271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.082725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.094413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 255/512] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:14.094810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 258/513] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:58:13 2022] POST /identity/v3/users => generated 247 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:14.095216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.095613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428975bb-b106-495f-9a3e-e53919318eb9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.128633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:14.224349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428975bb-b106-495f-9a3e-e53919318eb9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.224596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f027e13-b91b-4bd4-954a-361f1c32f043 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.252334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:14.266039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.266807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.266971 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.267107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.267341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.267622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.267901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.268170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.275330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.276826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.279355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.280838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:14.291276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:14.292167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:14.293045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 259/514] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:14.293487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:14.297908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:14.298761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:14.299582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:14.299582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:14.299582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:14.299582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 256/515] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:14.300222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.396466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:14.410475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619db019-94ed-410b-bf3f-05bb70aa9fbb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.410650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c69363-c207-4073-bc2a-f2e133effd65 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.445166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.445644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.445901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.446175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.446291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.446445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.446535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.446795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.447720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:14.448027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.450177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.454816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.456303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:14.468715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:14.469685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:14.470609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 257/516] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:14.471148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.504096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 260/517] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:14.504096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.521097 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:14.591653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a77f781-0e59-478a-b706-84b9c4cbaa4d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.604502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e898173-74ff-4493-8315-1053a198fef3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.628499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.629383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.629489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.629745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.630456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:14.630762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.632246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.642755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.643908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.644209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.644485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.645144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:14.645439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.647389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.650046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 258/518] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:14.650877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.661103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 261/519] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:14.662117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.674338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:14.751365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:14.758876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fb95d9-6cc2-4c3e-92dd-28e04f64bb64 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.773389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affcc86-a243-4c18-9015-8b7d916d40f8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.800388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.801056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.801570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.801865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.803242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:14.803543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.804983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.807032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e219-d454-48a3-b3ea-7e3cb653149e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.807624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e219-d454-48a3-b3ea-7e3cb653149e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.807896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e219-d454-48a3-b3ea-7e3cb653149e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.808161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e219-d454-48a3-b3ea-7e3cb653149e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.828197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 259/520] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:14.828197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:14.884038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49e219-d454-48a3-b3ea-7e3cb653149e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.885490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49e219-d454-48a3-b3ea-7e3cb653149e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.895426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:14.924377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a1b6b8-04eb-49ba-882d-ca3e15f56eb0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:14.957430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:14.958250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:14.958676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:14.959095 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:14.960340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=83f4897f360f4f23aac5749685fdf500) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:14.960817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:14.962755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:14.969048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 260/521] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/role_assignments?user.id=83f4897f360f4f23aac5749685fdf500&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:14.969048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.002845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:15.010823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 262/522] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:15.011712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e219-d454-48a3-b3ea-7e3cb653149e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.094618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160026f-e707-40ab-8612-e5bae50bda55 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.124685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e219-d454-48a3-b3ea-7e3cb653149e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.133107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:15.133310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.134259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.134741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.135201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.143530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.145420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:15.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:15.158937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 13:58:15.160114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:15.162653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 261/523] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:15.163698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.167414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb6be9-f3e6-4428-b869-599b89719bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.168050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb6be9-f3e6-4428-b869-599b89719bf2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.168330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb6be9-f3e6-4428-b869-599b89719bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.168690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb6be9-f3e6-4428-b869-599b89719bf2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.244303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:15.247205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb6be9-f3e6-4428-b869-599b89719bf2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.249487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb6be9-f3e6-4428-b869-599b89719bf2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.268869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3148fa-4dd8-4e80-896a-2760f83dd265 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.305308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.306002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.306277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.306550 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.318889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.320552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:15.335985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:15.336902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:15.337831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:15.337831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:15.337831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:15.337831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 262/524] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:15.338447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.360488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 263/525] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:15.362022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb6be9-f3e6-4428-b869-599b89719bf2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.379891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:15.462081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb6be9-f3e6-4428-b869-599b89719bf2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.468349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e66555e-7ab3-49ea-897c-570c1b85dac4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.500159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.501012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:15.502072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.502532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.502917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.503835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:15.504333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.506593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.517090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.517867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.518203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.518556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.519565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:15.519834 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.521366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.521919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 264/526] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:15.522900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.539956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 263/527] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:15.545219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.642044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:15.646179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69c907-27cc-432c-afc5-e036a6a75a5d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.665277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a126af-e60e-4223-a7eb-ead7b9b1f377 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.682509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.683376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.683683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.684040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.695005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.696686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.709032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.709940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.710465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.710869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:15.712423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:15.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:15.714584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:15.714584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:15.714584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:15.720337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 265/528] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:15.720337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.727045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.729248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:15.745764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:15.746677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:15.747513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:15.747513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:15.747513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:15.747513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:15.747513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:15.747513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:15.747513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:15.747513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:15.750167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 264/529] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:15.751106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:15.811906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:15.824133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a137ba-3e07-4b02-a8ae-8b4ef3417e74 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.842737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:15.863550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f9aa1-9c01-45be-ad1c-1f96f4a7b7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.864167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f9aa1-9c01-45be-ad1c-1f96f4a7b7ee None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.864439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f9aa1-9c01-45be-ad1c-1f96f4a7b7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.864701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207f9aa1-9c01-45be-ad1c-1f96f4a7b7ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.894922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b55b13b-ff6f-4926-836a-0c88526df33a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:15.936962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f9aa1-9c01-45be-ad1c-1f96f4a7b7ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.938651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207f9aa1-9c01-45be-ad1c-1f96f4a7b7ee None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.940580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:15.942256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:15.942531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:15.942797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:15.943440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:15.943737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:15.945111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:15.968057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 265/530] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:15.968865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.056877 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 266/531] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:16.057873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f9aa1-9c01-45be-ad1c-1f96f4a7b7ee None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.086146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:16.092532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a490f8de-0f75-4634-abe0-5d8dea953101 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:16.101187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:16.136209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:16.137612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.138173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.138608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.139493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:16.139962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:16.142473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:16.158404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 266/532] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:16.159679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.181473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207f9aa1-9c01-45be-ad1c-1f96f4a7b7ee None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:16.221943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:16.222772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.223111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.223433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.231340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:16.232917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:16.245940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:16.246852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:16.247677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:16.247677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:16.247677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:16.247677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:16.247677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:16.247677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:16.247677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:16.247677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:16.251636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 267/533] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:16.252750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.283156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0a6568-d043-4646-acd5-1a107bcacedf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:16.319099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:16.320103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.320617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.321147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.330694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:16.338115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:16.339514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:16.353351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:16.354262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:16.355465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:16.355465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:16.355465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:16.355465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 267/534] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:16.355886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94372df4-5672-45b7-91cb-2ca8095e59fb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:16.359656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.379187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:16.391003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89e70b5-4fec-477c-8d0b-95e75c36d2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:16.391762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e70b5-4fec-477c-8d0b-95e75c36d2eb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.392339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e70b5-4fec-477c-8d0b-95e75c36d2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.392612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89e70b5-4fec-477c-8d0b-95e75c36d2eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.393328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e70b5-4fec-477c-8d0b-95e75c36d2eb None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=44fba9b1749b4172bc9479ca980fcf48) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:16.393628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e70b5-4fec-477c-8d0b-95e75c36d2eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:16.395125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89e70b5-4fec-477c-8d0b-95e75c36d2eb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:16.400425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 268/535] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:16 2022] GET /identity/v3/role_assignments?user.id=44fba9b1749b4172bc9479ca980fcf48&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:16.401579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397afbb-4c17-4fef-8b89-3405675ac338 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.401881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397afbb-4c17-4fef-8b89-3405675ac338 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.402147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397afbb-4c17-4fef-8b89-3405675ac338 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.403513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 269/536] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:16.404009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397afbb-4c17-4fef-8b89-3405675ac338 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.490865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f0fef-ef8c-49f9-910c-f60eb3dab509 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:16.545483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397afbb-4c17-4fef-8b89-3405675ac338 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:16.549215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c6ace-b624-421b-885f-64bd5c40bbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:16.550131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6ace-b624-421b-885f-64bd5c40bbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.550582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6ace-b624-421b-885f-64bd5c40bbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.550983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c6ace-b624-421b-885f-64bd5c40bbe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.551842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c6ace-b624-421b-885f-64bd5c40bbe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:16.552288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c6ace-b624-421b-885f-64bd5c40bbe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:16.554836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c6ace-b624-421b-885f-64bd5c40bbe2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:16.571702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 268/537] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:16.573967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19147a-d8b1-4e24-8a7f-3bea75f8031e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.574634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19147a-d8b1-4e24-8a7f-3bea75f8031e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.575040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19147a-d8b1-4e24-8a7f-3bea75f8031e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.584570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:16.585757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.586204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.586531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.587202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:16.587503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:16.588925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:16.599253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c19147a-d8b1-4e24-8a7f-3bea75f8031e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:16.600444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c19147a-d8b1-4e24-8a7f-3bea75f8031e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:16.604953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 270/538] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:16.606359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.619970 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:16.702002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 269/539] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:16.706944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19147a-d8b1-4e24-8a7f-3bea75f8031e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.718901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:16.738910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1bd669-2056-4053-903c-58546c29f6ea None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:16.811199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:16.812189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.812658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.813902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.814847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=417b661f770846f189c36d0804b6dce5) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:16.817165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:16.820043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:16.831958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 271/540] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:16 2022] GET /identity/v3/role_assignments?user.id=417b661f770846f189c36d0804b6dce5&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:16.845922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19147a-d8b1-4e24-8a7f-3bea75f8031e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:16.883420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:16.884467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:16.884939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:16.885397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:16.893923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:16.895925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:16.908905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:16.909977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:16.910835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:16.910835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:16.910835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:16.910835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:16.910835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:16.910835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:16.910835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:16.910835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:16.913154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 270/541] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:16.919560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:16.996031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:17.035244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66721d9-2dc7-4f18-9fc7-01f71f5df425 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:17.060010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:17.072004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459124c-f25b-4039-acef-94244051a754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:17.072681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459124c-f25b-4039-acef-94244051a754 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:17.077726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459124c-f25b-4039-acef-94244051a754 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:17.077726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459124c-f25b-4039-acef-94244051a754 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:17.077726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459124c-f25b-4039-acef-94244051a754 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:17.077726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459124c-f25b-4039-acef-94244051a754 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:17.077726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c459124c-f25b-4039-acef-94244051a754 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:17.095056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 272/542] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:17.100943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459124c-f25b-4039-acef-94244051a754 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:17.197391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:17.247423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d36eee9-3024-4294-a0af-593827ae4923 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:17.271822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459124c-f25b-4039-acef-94244051a754 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:17.308380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c82fefd-4909-49cb-bd2e-d92f5cd04eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:17.309030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82fefd-4909-49cb-bd2e-d92f5cd04eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:17.309308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82fefd-4909-49cb-bd2e-d92f5cd04eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:17.309575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c82fefd-4909-49cb-bd2e-d92f5cd04eb5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:17.315447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:17.320729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c82fefd-4909-49cb-bd2e-d92f5cd04eb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:17.326027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2c82fefd-4909-49cb-bd2e-d92f5cd04eb5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:17.326426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:17.327457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:17.328442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:17.329148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:17.329148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:17.329148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:17.329148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=6c9b940cd9e0432daa235f82becb4d38) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:17.329148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:17.329673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:17.331040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 271/543] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:17.332461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361754e-f374-429e-873d-c3908c0c62d5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:17.332867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361754e-f374-429e-873d-c3908c0c62d5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:17.333259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361754e-f374-429e-873d-c3908c0c62d5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:17.338802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 273/545] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:17 2022] GET /identity/v3/role_assignments?user.id=6c9b940cd9e0432daa235f82becb4d38&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:17.339281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 272/545] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:17.341771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:17.342363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eed72-9cf0-45ca-8f13-e064c744ea22 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:17.342782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eed72-9cf0-45ca-8f13-e064c744ea22 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:17.343213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eed72-9cf0-45ca-8f13-e064c744ea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:17.370999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1f9eed72-9cf0-45ca-8f13-e064c744ea22 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:17.372473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1f9eed72-9cf0-45ca-8f13-e064c744ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:17.466006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dac031-d090-428f-85bb-e64da8141fc3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:17.517303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:17.518125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:17.518403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:17.519623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:17.520334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:17.520642 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:17.522070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:17.541761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 273/546] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:17.543907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 274/547] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:17.549685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9eed72-9cf0-45ca-8f13-e064c744ea22 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:17.632385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:17.676775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9eed72-9cf0-45ca-8f13-e064c744ea22 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:17.727594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:17.728258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:17.728531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:17.728797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:17.739928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:17.741525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:17.760329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:17.761281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:17.762935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:17.762935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:17.762935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:17.762935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 274/548] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:17.767147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:17.822241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:17.848849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:17.929388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69361fcd-1090-42dc-9038-16c64a16e253 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:17.992816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dee264-6df7-40da-875b-21bebacba85d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:18.003053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:18.003721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.003996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.004265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.004909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:18.005210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:18.006595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:18.010751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:18.032928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 275/549] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:18.035577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc085eb9-e43f-467c-b7ba-845ff22d36c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:18.036484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc085eb9-e43f-467c-b7ba-845ff22d36c1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.036958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc085eb9-e43f-467c-b7ba-845ff22d36c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.037503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc085eb9-e43f-467c-b7ba-845ff22d36c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.038255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:18.057855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc085eb9-e43f-467c-b7ba-845ff22d36c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-bc085eb9-e43f-467c-b7ba-845ff22d36c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:18.067962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:18.069837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:18.070793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 275/550] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:18.076272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc085eb9-e43f-467c-b7ba-845ff22d36c1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:18.234732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc085eb9-e43f-467c-b7ba-845ff22d36c1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:18.251851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfc5d6-1a57-4db0-83f2-39db1cb57f5d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:18.275707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:18.298586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:18.299211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.299482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.299747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.300443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:18.300738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:18.303025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:18.314014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:18.314653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.314923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.315188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.319810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 276/551] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:18.326382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:18.327072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:18.328522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:18.355953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:18.358400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:18.360573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 276/552] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:18.368881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:18.389501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:18.524267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d6bf1-38dc-4a99-b7e7-b16e7c7f7395 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:18.558793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08064b-7b7e-462f-969f-7b0d0d36b195 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:18.596792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:18.597432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.597745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.598015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.598741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:18.599043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:18.601937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:18.605727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:18.606381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.606655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.606921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.627158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:18.628654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:18.642320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 277/553] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 13:58:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:18.649928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:18.650949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:18.651821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:18.651821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:18.651821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:18.651985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:18.652064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 277/554] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:18.657157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:18.705864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:18.812929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:18.832700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14461c93-1feb-49c7-8b36-681dc552bd03 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:18.844011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7e073-4eeb-4be3-87a7-668d24b48b33 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:18.881290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046c1fab-df74-45fa-8aef-0b34e6d18a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:18.882043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c1fab-df74-45fa-8aef-0b34e6d18a53 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.882327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c1fab-df74-45fa-8aef-0b34e6d18a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.882597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046c1fab-df74-45fa-8aef-0b34e6d18a53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.886169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e7aa6-a98f-4732-acc4-1d2f258b8cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:18.887050 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e7aa6-a98f-4732-acc4-1d2f258b8cdd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.887524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e7aa6-a98f-4732-acc4-1d2f258b8cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.887987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e7aa6-a98f-4732-acc4-1d2f258b8cdd None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.888852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e7aa6-a98f-4732-acc4-1d2f258b8cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:18.889351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e7aa6-a98f-4732-acc4-1d2f258b8cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:18.892068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529e7aa6-a98f-4732-acc4-1d2f258b8cdd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:18.906234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 278/555] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:18.907879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb448892-ede0-4eb1-af56-1822db35eef2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:18.908494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb448892-ede0-4eb1-af56-1822db35eef2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:18.908992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb448892-ede0-4eb1-af56-1822db35eef2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:18.914756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 279/556] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:18.915860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb448892-ede0-4eb1-af56-1822db35eef2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:18.958354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c1fab-df74-45fa-8aef-0b34e6d18a53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:18.960307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046c1fab-df74-45fa-8aef-0b34e6d18a53 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:18.993865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:19.027416 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb448892-ede0-4eb1-af56-1822db35eef2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:19.065401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:19.066525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.067020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.067535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.073799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 278/557] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:19.075172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4775-e748-40d3-9045-a30d54e324a5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.075452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4775-e748-40d3-9045-a30d54e324a5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.075722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240b4775-e748-40d3-9045-a30d54e324a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.076729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:19.078921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:19.092100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:19.093104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:19.094069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:19.095098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-240b4775-e748-40d3-9045-a30d54e324a5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:19.096034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-240b4775-e748-40d3-9045-a30d54e324a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:19.096959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 280/558] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:58:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:19.098137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.102158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:19.185661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 279/559] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:19.185661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4775-e748-40d3-9045-a30d54e324a5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.239975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decbaebb-7446-4326-add8-af74af049cf0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:19.255293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:19.306688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:19.307336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.307608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.307880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.317115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240b4775-e748-40d3-9045-a30d54e324a5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:19.335629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:19.337107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:19.359543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:19.360548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:19.361825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:19.361825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:19.361825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:19.361825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 281/560] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:19.365674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a97a5b-31b0-4b10-abbe-693748fe591f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:19.365674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a97a5b-31b0-4b10-abbe-693748fe591f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.365674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a97a5b-31b0-4b10-abbe-693748fe591f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.365674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a97a5b-31b0-4b10-abbe-693748fe591f None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.366024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a97a5b-31b0-4b10-abbe-693748fe591f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:19.366024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a97a5b-31b0-4b10-abbe-693748fe591f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:19.366629 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a97a5b-31b0-4b10-abbe-693748fe591f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:19.369260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.390153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 280/561] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:19.391446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475e724-293f-4989-ac83-e504aeae9989 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.391724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475e724-293f-4989-ac83-e504aeae9989 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.391995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8475e724-293f-4989-ac83-e504aeae9989 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.397615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 281/562] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:19 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:19.398955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbd9a-2481-4689-bf93-8e0bc3eda2ec None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.399232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbd9a-2481-4689-bf93-8e0bc3eda2ec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.399499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202cbd9a-2481-4689-bf93-8e0bc3eda2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.419215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-202cbd9a-2481-4689-bf93-8e0bc3eda2ec None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:19.420405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-202cbd9a-2481-4689-bf93-8e0bc3eda2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:19.441772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:19.498692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db09cb-a94b-4bf2-a49c-fe549cb50c29 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:19.509093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 282/563] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:19.510135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cbd9a-2481-4689-bf93-8e0bc3eda2ec None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.548767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:19.549556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.549962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.550595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.551754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:19.552161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:19.553942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:19.570546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 282/564] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:19.571780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:19.572435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.634841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202cbd9a-2481-4689-bf93-8e0bc3eda2ec None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:19.683691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:19.685053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.685682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.686039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.691472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d16b2ef-7a9f-4c58-bd67-4531d85aa1f0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:19.701450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:19.703130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:19.711539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:19.727055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:19.728095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:19.733720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:19.733720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:19.733720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:19.733720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 283/565] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:19.733720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.739346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151744-b378-4717-98de-33fe077dc263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:19.740278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151744-b378-4717-98de-33fe077dc263 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.740749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151744-b378-4717-98de-33fe077dc263 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.741222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151744-b378-4717-98de-33fe077dc263 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.753581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd151744-b378-4717-98de-33fe077dc263 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-cd151744-b378-4717-98de-33fe077dc263 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:19.758860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:19.759793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:19.760620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:19.767897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 283/566] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:19.769146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151744-b378-4717-98de-33fe077dc263 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.839218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:19.842553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca951a19-372e-4ef3-b60f-05e15f301604 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:19.869969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151744-b378-4717-98de-33fe077dc263 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:19.877774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:19.878408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.878744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.879008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.885929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:19.887366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:19.898479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:19.899347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:19.900218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 284/567] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:19.900693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.906591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:19.907204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:19.907477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:19.907742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:19.908378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:19.908674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:19.910141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:19.921496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 284/568] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:19.922324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:19.959214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.000333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec3f46f-f5b9-4d34-87b3-3e0a4c8c3598 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.022661 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46634897-4100-4c33-b0b7-8f884b3df6f7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.034514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c5ae96-9bfa-4ce5-b6ad-f9a22c111132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.035161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ae96-9bfa-4ce5-b6ad-f9a22c111132 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.035437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ae96-9bfa-4ce5-b6ad-f9a22c111132 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.035704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c5ae96-9bfa-4ce5-b6ad-f9a22c111132 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.036366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5ae96-9bfa-4ce5-b6ad-f9a22c111132 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:20.036665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5ae96-9bfa-4ce5-b6ad-f9a22c111132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.038050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c5ae96-9bfa-4ce5-b6ad-f9a22c111132 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.045871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 285/569] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:20.047082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde232-9de0-4fe9-87fb-1290a5053435 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.047386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde232-9de0-4fe9-87fb-1290a5053435 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.047627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fde232-9de0-4fe9-87fb-1290a5053435 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.048946 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 286/570] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:20.049542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fde232-9de0-4fe9-87fb-1290a5053435 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.059120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa522a-8a2b-4f96-af47-f40f4f307238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.059767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa522a-8a2b-4f96-af47-f40f4f307238 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.060052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa522a-8a2b-4f96-af47-f40f4f307238 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.060324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aa522a-8a2b-4f96-af47-f40f4f307238 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.070530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aa522a-8a2b-4f96-af47-f40f4f307238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:20.072891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-45aa522a-8a2b-4f96-af47-f40f4f307238 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:20.075800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:20.077252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 285/571] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:20.078176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa522a-8a2b-4f96-af47-f40f4f307238 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.156060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fde232-9de0-4fe9-87fb-1290a5053435 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.169213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.175723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aa522a-8a2b-4f96-af47-f40f4f307238 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.189348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.189985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.190260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.190528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.191283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:20.191584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.193103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.206733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 287/572] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:20.207301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.213818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be712d-bfde-4319-88e1-ecd6e1868dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.214474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be712d-bfde-4319-88e1-ecd6e1868dae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.214748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be712d-bfde-4319-88e1-ecd6e1868dae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.215011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be712d-bfde-4319-88e1-ecd6e1868dae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.222060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be712d-bfde-4319-88e1-ecd6e1868dae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.223957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be712d-bfde-4319-88e1-ecd6e1868dae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-71be712d-bfde-4319-88e1-ecd6e1868dae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:20.237672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:20.238766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 286/573] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:20.240270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374d5a-45e8-4c99-83b5-b90c4915cb9a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.240951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374d5a-45e8-4c99-83b5-b90c4915cb9a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.240951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d374d5a-45e8-4c99-83b5-b90c4915cb9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.259271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2d374d5a-45e8-4c99-83b5-b90c4915cb9a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:20.260468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2d374d5a-45e8-4c99-83b5-b90c4915cb9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:20.292388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.311848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754f62d-b361-4702-abaf-dcf9a4b5ca99 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.327888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 287/574] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:20.328558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d374d5a-45e8-4c99-83b5-b90c4915cb9a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.346246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.346883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.347190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.347487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.358592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.360004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:20.375302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:20.376428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:20.377901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:20.377901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:20.377901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:20.377901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 288/575] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:20.378817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.419344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.435808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d374d5a-45e8-4c99-83b5-b90c4915cb9a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.470874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.471482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.471782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.472063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.472828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:20.473134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.474549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.482599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 288/576] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:20.483396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.499468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62516e4a-193e-4bb4-a060-de913fca539e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.539341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.539964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.540242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.540511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.541251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=01286f6f65b44519b645927f6c70ec5b) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:20.541560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.543043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.544923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.549733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 289/577] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/role_assignments?user.id=01286f6f65b44519b645927f6c70ec5b&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:20.550673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.622579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af4801-91b9-4584-afe9-6ab4a3b90ba9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.661650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.662341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.662613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.662875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.662999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94845c1c-0cad-4b9c-ada4-33cbf135b294 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.663583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:20.663881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.665374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.678558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 289/578] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:20.679339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.692292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.712245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.713446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.714316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.714879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.715786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:20.716258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.718029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.735010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 290/579] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:20.735910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.791232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bcddbb-1c75-4cb9-a220-2969aa75a8f5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.802397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.826373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.827038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.827315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.827591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.837939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:20.839402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:20.846634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d90c6-ac3f-465a-add2-c8e124290221 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:20.854219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:20.855333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:20.856408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:20.856408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:20.856408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:20.856408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 290/580] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:20.858092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.886331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c642af9-dda4-474c-803c-bc95bf14dd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:20.886982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c642af9-dda4-474c-803c-bc95bf14dd2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:20.887259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c642af9-dda4-474c-803c-bc95bf14dd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:20.887525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c642af9-dda4-474c-803c-bc95bf14dd2b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:20.901517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c642af9-dda4-474c-803c-bc95bf14dd2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7c642af9-dda4-474c-803c-bc95bf14dd2b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:20.907635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:20.908577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:20.909489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:20.910203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 291/581] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:20.910768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c642af9-dda4-474c-803c-bc95bf14dd2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:20.930956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:20.974750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4105e541-0c4a-4071-88d7-306f81b1b79f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.008715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d852b-c233-462a-bde5-63ba9ea973c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.009406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d852b-c233-462a-bde5-63ba9ea973c8 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.009707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d852b-c233-462a-bde5-63ba9ea973c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.009979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067d852b-c233-462a-bde5-63ba9ea973c8 None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/83f4897f360f4f23aac5749685fdf500/roles/7d39e08123cb40c78586e4d7eb3ed9ba` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.019632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c642af9-dda4-474c-803c-bc95bf14dd2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.025217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d852b-c233-462a-bde5-63ba9ea973c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=83f4897f360f4f23aac5749685fdf500, role_id=7d39e08123cb40c78586e4d7eb3ed9ba)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.026825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067d852b-c233-462a-bde5-63ba9ea973c8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.040351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:21.043383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 291/582] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:58:20 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/83f4897f360f4f23aac5749685fdf500/roles/7d39e08123cb40c78586e4d7eb3ed9ba => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:21.045230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d852b-c233-462a-bde5-63ba9ea973c8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.072488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.074222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.074613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.074956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.086886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.088732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:21.105388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:21.106475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 13:58:21.107407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:21.109818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 292/583] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:21.111001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.178079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067d852b-c233-462a-bde5-63ba9ea973c8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.194470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:21.231062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.231466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.231741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.232004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.232650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:21.232945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.234363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.247473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 292/584] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:21.248428 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.255855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90882a19-188b-4f76-a042-28ea779111fa None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.290895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.291873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.292397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.292814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.293486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:21.293819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.295185 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.304326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 293/585] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:21.305014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.349512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:21.380376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d1b39-c217-4d00-8441-8e296fd6b150 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.413485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1346c1-d6e0-41a6-bb8f-5e605b5c41e6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.426201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.427229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.427657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.428099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.429826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:21.430201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.432744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.456681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.457667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.458300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 293/586] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:21.458643 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.459053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.459501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.469574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.472134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:21.485680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:21.487188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 05 13:58:21.488384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:21.490829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:21.496073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 294/587] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:21.497264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.593847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d766f7-82e7-49e5-a801-11cef27e984b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.607425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:21.611999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70e177-217b-4ade-862a-591a757ad17f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.640598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.641367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.641719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.641995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.648404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.649342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.649893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.650356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.654906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.656436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.662123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.663735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:21.675331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:21.676281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:21.677137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:21.677137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:21.677137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:21.677137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 294/588] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:21.678212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:21.681478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:21.682467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:21.683328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:21.683328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:21.683328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:21.689673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 295/589] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:21.690866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.796075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af4f246-ec1f-459c-82f1-9d64a61599ba None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.814074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:21.817291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abba047-b7d3-41a4-879b-f8822a712e9e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.831453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.832114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.832389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.832652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.833377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:21.833715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.835373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.857895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 295/590] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:21.859549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.866167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:21.866913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:21.867251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:21.867545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:21.868193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:21.868488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:21.869935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:21.883402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 296/591] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:21.884230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:21.900149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:21.969739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f38aa-94a1-44bc-af3b-317b355ddaf8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:21.993105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a212412-3056-4a09-9d82-a440e37766db None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.008054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.009008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.009496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.009943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.010729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:22.011193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.012887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.020072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:22.026909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 296/592] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:22.027955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.035044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.035890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.036301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.036744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.048596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.050306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:22.066100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:22.067197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:22.068243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:22.068243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:22.068243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:22.069680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 297/593] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:22.070614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.135577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:22.139064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7684abd5-f847-4e42-8975-bb052b08b50c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.175600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.176133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.176415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.176739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.179142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb06dba-efc3-4fd9-99a0-3b8a535c7681 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.184833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.186307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:22.198459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:22.199498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:22.200633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 297/594] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:22.201325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.224072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.225420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.226079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.226698 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.238572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.241247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:22.257562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:22.258613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:22.259594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:22.259594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:22.259594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:22.259594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:22.259594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:22.259594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:22.259594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:22.259594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:22.262575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 298/595] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:22.264110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.266529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:22.311820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b13575-c76b-4828-90aa-54fd8dd43b48 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.347707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.348317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.348593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.348859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.349545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:22.349875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.351512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.367380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 298/596] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:22.372637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.391390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d3c534-0c13-4c01-9485-c0b1f3a3061d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.416570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:22.428179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.428919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.429289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.429662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.430501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:22.430901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.432590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.451878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 299/597] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:22.452926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.480863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a0f78a-011d-473f-9033-cd57b5a3e7ef None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.499903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:22.518244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.519006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.519285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.519603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.520292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:22.520641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.522320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.535523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 299/598] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:22.536453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.573236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31b525f-8071-4d80-b39e-8434e02f2f0b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.615158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.615975 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.616345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.616747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.627897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.629625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:22.642511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:22.644222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:22.645991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:22.645991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:22.645991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:22.645991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:22.645991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:22.645991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:22.645991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:22.645991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:22.653690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 300/599] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:22.653690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.661785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a1139b-2621-4d79-bbce-8fa13682e064 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.693584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:22.699343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.701616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.701616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.701616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.713707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.715531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:22.730935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:22.732058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:22.733137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:22.733137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:22.733137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:22.733137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 300/600] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:22.733137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.755744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9471db7c-73b0-49f8-923e-40cac8ebbe60 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.765315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:22.791498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be41458-a33d-4e39-982c-208e65c73e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.792251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be41458-a33d-4e39-982c-208e65c73e78 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.792598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be41458-a33d-4e39-982c-208e65c73e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.792935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be41458-a33d-4e39-982c-208e65c73e78 None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/44fba9b1749b4172bc9479ca980fcf48/roles/7d39e08123cb40c78586e4d7eb3ed9ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.807938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be41458-a33d-4e39-982c-208e65c73e78 None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=44fba9b1749b4172bc9479ca980fcf48, role_id=7d39e08123cb40c78586e4d7eb3ed9ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.809544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be41458-a33d-4e39-982c-208e65c73e78 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.824288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 301/601] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:58:22 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/44fba9b1749b4172bc9479ca980fcf48/roles/7d39e08123cb40c78586e4d7eb3ed9ba => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:22.825364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d5a13-358e-47fa-ac25-13ad515a5fb0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.825668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d5a13-358e-47fa-ac25-13ad515a5fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.825948 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d5a13-358e-47fa-ac25-13ad515a5fb0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.831763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 302/602] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:22.831763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d5a13-358e-47fa-ac25-13ad515a5fb0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:22.853544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a11ba20-2396-456a-a828-f49c448aa092 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:22.900349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e620f9-85ca-4f6c-9a90-c5015e583dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:22.901739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e620f9-85ca-4f6c-9a90-c5015e583dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.902436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e620f9-85ca-4f6c-9a90-c5015e583dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.902940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e620f9-85ca-4f6c-9a90-c5015e583dc4 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.903831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e620f9-85ca-4f6c-9a90-c5015e583dc4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:22.904624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e620f9-85ca-4f6c-9a90-c5015e583dc4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:22.907274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e620f9-85ca-4f6c-9a90-c5015e583dc4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:22.926755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 301/603] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:22.928520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e501f-3ba9-4025-a900-a0a6c0ae02fc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:22.928942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e501f-3ba9-4025-a900-a0a6c0ae02fc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:22.929357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748e501f-3ba9-4025-a900-a0a6c0ae02fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:22.955569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-748e501f-3ba9-4025-a900-a0a6c0ae02fc None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:22.957491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-748e501f-3ba9-4025-a900-a0a6c0ae02fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:22.970369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:22.984940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d5a13-358e-47fa-ac25-13ad515a5fb0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:23.028277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:23.028903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.029182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.029450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.030196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:23.030500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:23.032038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:23.042971 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 302/604] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:23.043848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e501f-3ba9-4025-a900-a0a6c0ae02fc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:23.045443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 303/605] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:23.056671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:23.089290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:23.177229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1e5b07-55cc-4a27-bc50-37be04306d9f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:23.183411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748e501f-3ba9-4025-a900-a0a6c0ae02fc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:23.213961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:23.214651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.214874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.215144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.217879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a658c3-86b6-4e69-b4f6-96bfbdca552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:23.218546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a658c3-86b6-4e69-b4f6-96bfbdca552e None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.218818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a658c3-86b6-4e69-b4f6-96bfbdca552e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.219082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a658c3-86b6-4e69-b4f6-96bfbdca552e None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/417b661f770846f189c36d0804b6dce5/roles/7d39e08123cb40c78586e4d7eb3ed9ba` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.222452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:23.223901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:23.234023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a658c3-86b6-4e69-b4f6-96bfbdca552e None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=417b661f770846f189c36d0804b6dce5, role_id=7d39e08123cb40c78586e4d7eb3ed9ba)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:23.235534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a658c3-86b6-4e69-b4f6-96bfbdca552e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:23.235915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:23.237143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:23.238123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 304/606] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:23.245948 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:23.257580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 303/607] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:58:23 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/417b661f770846f189c36d0804b6dce5/roles/7d39e08123cb40c78586e4d7eb3ed9ba => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:23.268790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:23.399643 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595f435-3847-4791-a801-0579a7aecd91 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:23.417675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc02db-7a52-4981-9d11-26cd143de272 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.417970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc02db-7a52-4981-9d11-26cd143de272 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.418239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc02db-7a52-4981-9d11-26cd143de272 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.419998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 304/608] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:58:23.433549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc02db-7a52-4981-9d11-26cd143de272 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:23.439968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9620c61f-5f1d-46ae-ae3b-87837f6045ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:23.440841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620c61f-5f1d-46ae-ae3b-87837f6045ff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.441245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620c61f-5f1d-46ae-ae3b-87837f6045ff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.441620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9620c61f-5f1d-46ae-ae3b-87837f6045ff None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.442477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620c61f-5f1d-46ae-ae3b-87837f6045ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:23.442888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620c61f-5f1d-46ae-ae3b-87837f6045ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:23.444626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9620c61f-5f1d-46ae-ae3b-87837f6045ff None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:23.460220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 305/609] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:23.461891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978f712-2a06-4505-8a2f-c20a065231f9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.462280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978f712-2a06-4505-8a2f-c20a065231f9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.462651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978f712-2a06-4505-8a2f-c20a065231f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.482544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3978f712-2a06-4505-8a2f-c20a065231f9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:23.484137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3978f712-2a06-4505-8a2f-c20a065231f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:23.524703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:23.567397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 306/610] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:23.568003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc02db-7a52-4981-9d11-26cd143de272 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:23.569273 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978f712-2a06-4505-8a2f-c20a065231f9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:23.573087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:23.645476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c9208-4a2a-4d02-9cd3-0b8aef5ff81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:23.646434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c9208-4a2a-4d02-9cd3-0b8aef5ff81c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.646739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c9208-4a2a-4d02-9cd3-0b8aef5ff81c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.647109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5c9208-4a2a-4d02-9cd3-0b8aef5ff81c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.678616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5c9208-4a2a-4d02-9cd3-0b8aef5ff81c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0d5c9208-4a2a-4d02-9cd3-0b8aef5ff81c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:23.683585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:23.684685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:23.685690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:23.697739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 305/611] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:23.698059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978f712-2a06-4505-8a2f-c20a065231f9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:23.707952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c9208-4a2a-4d02-9cd3-0b8aef5ff81c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:23.746146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6739ec-97e0-4d00-beb7-38a8797bb804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:23.748038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6739ec-97e0-4d00-beb7-38a8797bb804 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.749536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6739ec-97e0-4d00-beb7-38a8797bb804 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.749973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6739ec-97e0-4d00-beb7-38a8797bb804 None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/6c9b940cd9e0432daa235f82becb4d38/roles/7d39e08123cb40c78586e4d7eb3ed9ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.780303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6739ec-97e0-4d00-beb7-38a8797bb804 None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=6c9b940cd9e0432daa235f82becb4d38, role_id=7d39e08123cb40c78586e4d7eb3ed9ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:23.782680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6739ec-97e0-4d00-beb7-38a8797bb804 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:23.807086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 307/612] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:58:23 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/6c9b940cd9e0432daa235f82becb4d38/roles/7d39e08123cb40c78586e4d7eb3ed9ba => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:23.868484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5c9208-4a2a-4d02-9cd3-0b8aef5ff81c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:23.901592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6739ec-97e0-4d00-beb7-38a8797bb804 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:23.905626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:23.912106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:23.912728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:23.912999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:23.913265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:23.913975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:23.914271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:23.915651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:23.929434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 306/613] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:23.933515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:24.022058 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:24.098317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6739ec-97e0-4d00-beb7-38a8797bb804 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:24.110783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b9b0a-c9c0-4725-8d2a-bd007bd890a4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:24.137373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36197d07-a279-4c00-8217-529c4a858d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:24.138063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36197d07-a279-4c00-8217-529c4a858d16 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:24.138341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36197d07-a279-4c00-8217-529c4a858d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:24.138614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36197d07-a279-4c00-8217-529c4a858d16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:24.151573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36197d07-a279-4c00-8217-529c4a858d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:24.153051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:24.153783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:24.154160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:24.154491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-36197d07-a279-4c00-8217-529c4a858d16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:24.156373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:24.157278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:24.158172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 308/614] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:24.163447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36197d07-a279-4c00-8217-529c4a858d16 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:24.175736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:24.177281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:24.200949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:24.203226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:24.205374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:24.205374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:24.205374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:24.205374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 307/615] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:24.211839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:24.295103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:24.350231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36197d07-a279-4c00-8217-529c4a858d16 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:24.386021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:24.397844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:24.398541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:24.398887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:24.399218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:24.399950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:24.400483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:24.401818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:24.410953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 309/616] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:24.417090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:24.424730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7de05e-0607-4ddf-96cf-0dc6c9aab790 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:24.485808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:24.486700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:24.486968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:24.487230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:24.488115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:24.489659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:24.491044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:24.517984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 308/617] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:24.523482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:24.611856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec972adf-da6f-4c39-b844-1bba7d8ec6de None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:24.671352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:24.672053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:24.672320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:24.672578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:24.690818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:24.692273 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:24.695353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ff7cee-9a24-4dcb-b059-16ff4e5c4020 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:24.698756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:24.720518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:24.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:24.722870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:24.722870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:24.722870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:24.722870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 310/618] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:24.727585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:24.766589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:24.766589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:24.766589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:24.766589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:24.778557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:24.780306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:24.784774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:24.812072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:24.813345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:24.814529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:24.817666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 309/619] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:24.822152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:24.891945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ee1712-70fc-49f4-ae38-c624550b4eca None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:24.927821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:24.928314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:24.928655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:24.928986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:24.929791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:24.930143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:24.937676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:24.946683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 311/620] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:24.951896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:25.045958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:25.049988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd7fb86-3614-49a2-9eba-f06f6f8b7037 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:25.062969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb5051-dab9-44ab-8583-8c286b6bebd3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:25.098151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:25.098786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.099100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.099367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.106835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:25.108429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:25.121269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:25.122254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:25.123178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 312/621] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:58:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:25.128315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:25.147215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:25.147835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.148108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.148372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.149157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:25.149458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:25.150926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:25.185129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 310/622] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 13:58:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:25.190448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:25.233874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:25.235889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d0ba52-1173-411c-99fb-92db8836efd8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:25.281251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960f405-b095-4760-aff8-136156522c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:25.282336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f405-b095-4760-aff8-136156522c6f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.282959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f405-b095-4760-aff8-136156522c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.284244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8960f405-b095-4760-aff8-136156522c6f None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.285519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f405-b095-4760-aff8-136156522c6f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:25.287558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f405-b095-4760-aff8-136156522c6f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:25.289593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8960f405-b095-4760-aff8-136156522c6f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:25.324354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 313/623] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:58:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:25.349531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960f405-b095-4760-aff8-136156522c6f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:25.353977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:25.387821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e77c091-af60-451e-9105-1dd5298ba0b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:25.432875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158ec06-f81c-4390-8ae3-e06763c76266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:25.433520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ec06-f81c-4390-8ae3-e06763c76266 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.433823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ec06-f81c-4390-8ae3-e06763c76266 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.434089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ec06-f81c-4390-8ae3-e06763c76266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.444990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158ec06-f81c-4390-8ae3-e06763c76266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:25.447414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c158ec06-f81c-4390-8ae3-e06763c76266 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c158ec06-f81c-4390-8ae3-e06763c76266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:25.465088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:25.466154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:25.467075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:25.467075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:25.467075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:25.467075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 311/624] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:25.469662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b07d5c-e7d0-4765-b7ea-1c44ec02db35 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.469662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b07d5c-e7d0-4765-b7ea-1c44ec02db35 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.469662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b07d5c-e7d0-4765-b7ea-1c44ec02db35 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.469918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 312/625] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:25.470104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8960f405-b095-4760-aff8-136156522c6f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:25.476621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1006e-60be-406f-9b39-077db4b349b2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.476973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1006e-60be-406f-9b39-077db4b349b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.477244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1006e-60be-406f-9b39-077db4b349b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.496896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b2a1006e-60be-406f-9b39-077db4b349b2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:25.498220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b2a1006e-60be-406f-9b39-077db4b349b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:25.506472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2141b-62f5-4983-941b-002f446482a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:25.507217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2141b-62f5-4983-941b-002f446482a9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.507483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2141b-62f5-4983-941b-002f446482a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.507749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a2141b-62f5-4983-941b-002f446482a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.581769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2141b-62f5-4983-941b-002f446482a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:25.581769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a2141b-62f5-4983-941b-002f446482a9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:25.582452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:25.583304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 313/626] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:25.584494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1006e-60be-406f-9b39-077db4b349b2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:25.588986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:25.680662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 314/627] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 13:58:25.687086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2141b-62f5-4983-941b-002f446482a9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:25.796372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1006e-60be-406f-9b39-077db4b349b2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:25.811735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a2141b-62f5-4983-941b-002f446482a9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:25.820130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:25.867121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:25.867961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.869388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.870332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.871272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:25.872254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:25.874890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:25.887578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:25.888355 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:25.888721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:25.889084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:25.899342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 314/628] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:25.905144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:25.908794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:25.910410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:25.935543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:25.936838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:25.937864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:25.937864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:25.937864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:25.937864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 315/629] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:25.937864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:25.991046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:26.032817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9177fb-2f38-4fd6-a65d-98c7604455e5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.043469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415b2cc-79ec-4a6b-bbc4-e3840668e04b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.068002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.068740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.069016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.069280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.076758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:26.078440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:26.082788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5f9036-f4cc-4e7a-9a06-9a3fbe3d099d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.083439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f9036-f4cc-4e7a-9a06-9a3fbe3d099d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.083710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f9036-f4cc-4e7a-9a06-9a3fbe3d099d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.083971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5f9036-f4cc-4e7a-9a06-9a3fbe3d099d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:26.090917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:26.091789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:26.092698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 315/630] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:26.093201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.100879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5f9036-f4cc-4e7a-9a06-9a3fbe3d099d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ef5f9036-f4cc-4e7a-9a06-9a3fbe3d099d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:26.106054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:26.106976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:26.107819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 316/631] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:26.108543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93060a4-0990-484a-9a3b-b401f9b9f807 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.108623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93060a4-0990-484a-9a3b-b401f9b9f807 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.109160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93060a4-0990-484a-9a3b-b401f9b9f807 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.110355 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 317/632] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:26.111042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93060a4-0990-484a-9a3b-b401f9b9f807 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.133558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:26.199967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f4ae44-c961-4e85-9d72-8dd35d9adc86 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.227272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:26.233843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93060a4-0990-484a-9a3b-b401f9b9f807 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.241365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.242559 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.242837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.243103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.243749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:26.244410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:26.246193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:26.263542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 316/633] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:26.263542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.274147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af960d5-72b5-45ec-8d9c-046476bc3420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.274752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af960d5-72b5-45ec-8d9c-046476bc3420 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.275028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af960d5-72b5-45ec-8d9c-046476bc3420 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.275299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af960d5-72b5-45ec-8d9c-046476bc3420 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.275982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af960d5-72b5-45ec-8d9c-046476bc3420 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:26.276287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af960d5-72b5-45ec-8d9c-046476bc3420 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:26.277737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af960d5-72b5-45ec-8d9c-046476bc3420 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:26.293063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 318/634] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:26.294328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c591-4251-4778-97e3-882953470c83 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.294614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c591-4251-4778-97e3-882953470c83 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.294881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d6c591-4251-4778-97e3-882953470c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.313732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-64d6c591-4251-4778-97e3-882953470c83 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:26.314933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-64d6c591-4251-4778-97e3-882953470c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:26.371301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612a996b-ff3f-4683-8731-e8b3b216ee2b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.381144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 319/635] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:26.382032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c591-4251-4778-97e3-882953470c83 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.389896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:26.406293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.406868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.407147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.407417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.408085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:26.408383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:26.409868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:26.424360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 317/636] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:26.424360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.483417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:26.493461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d6c591-4251-4778-97e3-882953470c83 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.536530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d814e-1c37-4ab8-8941-f4edadd077e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.537320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d814e-1c37-4ab8-8941-f4edadd077e9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.537542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d814e-1c37-4ab8-8941-f4edadd077e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.537838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d814e-1c37-4ab8-8941-f4edadd077e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.541283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b3930c-db12-43bb-ad85-9005dc14e464 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.545203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d814e-1c37-4ab8-8941-f4edadd077e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:26.546694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d814e-1c37-4ab8-8941-f4edadd077e9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5d6d814e-1c37-4ab8-8941-f4edadd077e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:26.557937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:26.559032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 320/637] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:26.560570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62426ab1-f638-4b6c-8c46-7f4d77c2bcb0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.561132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62426ab1-f638-4b6c-8c46-7f4d77c2bcb0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.561132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62426ab1-f638-4b6c-8c46-7f4d77c2bcb0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.562504 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 321/638] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:26.567198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62426ab1-f638-4b6c-8c46-7f4d77c2bcb0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.581615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f7c9d-7f8e-45f0-909b-721f61a39450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.582314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7c9d-7f8e-45f0-909b-721f61a39450 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.582597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7c9d-7f8e-45f0-909b-721f61a39450 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.582959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f7c9d-7f8e-45f0-909b-721f61a39450 None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/01286f6f65b44519b645927f6c70ec5b/roles/7d39e08123cb40c78586e4d7eb3ed9ba` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.597914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f7c9d-7f8e-45f0-909b-721f61a39450 None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=01286f6f65b44519b645927f6c70ec5b, role_id=7d39e08123cb40c78586e4d7eb3ed9ba)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:26.599457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f7c9d-7f8e-45f0-909b-721f61a39450 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:26.614202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 318/639] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:58:26 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/01286f6f65b44519b645927f6c70ec5b/roles/7d39e08123cb40c78586e4d7eb3ed9ba => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:26.618534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f7c9d-7f8e-45f0-909b-721f61a39450 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.643704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:26.687002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62426ab1-f638-4b6c-8c46-7f4d77c2bcb0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.726178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf094d7-f6ab-4f54-8f4e-94222131682d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.727221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf094d7-f6ab-4f54-8f4e-94222131682d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.727690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf094d7-f6ab-4f54-8f4e-94222131682d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.728119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf094d7-f6ab-4f54-8f4e-94222131682d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.731863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:26.738534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f7c9d-7f8e-45f0-909b-721f61a39450 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.745684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf094d7-f6ab-4f54-8f4e-94222131682d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:26.748652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf094d7-f6ab-4f54-8f4e-94222131682d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6bf094d7-f6ab-4f54-8f4e-94222131682d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:26.769285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:26.770498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:26.771356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:26.771356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:26.771356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:26.771356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 322/640] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:26.771766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cb25a-3768-4ef9-859d-e029440fbeca None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.772050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cb25a-3768-4ef9-859d-e029440fbeca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.772316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cb25a-3768-4ef9-859d-e029440fbeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.789683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.790585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.791009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.791450 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.791562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7e8cb25a-3768-4ef9-859d-e029440fbeca None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:26.792403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:26.792686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7e8cb25a-3768-4ef9-859d-e029440fbeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:26.792788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:26.794215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:26.809177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 319/641] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:26.810040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.868439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 323/642] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:26.869406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cb25a-3768-4ef9-859d-e029440fbeca None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:26.922679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:26.932691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9295614-2906-4387-bad7-a751d43f5b47 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.969123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0a2c9-33b5-4cbf-8302-63e97a9ea747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:26.970173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0a2c9-33b5-4cbf-8302-63e97a9ea747 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:26.970760 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0a2c9-33b5-4cbf-8302-63e97a9ea747 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:26.971254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a0a2c9-33b5-4cbf-8302-63e97a9ea747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:26.980951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cb25a-3768-4ef9-859d-e029440fbeca None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:26.983143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a0a2c9-33b5-4cbf-8302-63e97a9ea747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-16a0a2c9-33b5-4cbf-8302-63e97a9ea747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:26.989038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:26.990081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:26.990928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:26.993887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 320/643] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:26.994978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0a2c9-33b5-4cbf-8302-63e97a9ea747 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.015725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:27.027919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.028922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.029205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.029473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.030183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:27.030858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.032351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.049809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 324/644] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:27.051387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.111051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a0a2c9-33b5-4cbf-8302-63e97a9ea747 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.148467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.149297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.149724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.150079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.151454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=83f4897f360f4f23aac5749685fdf500) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:27.151861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.153423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.160395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 321/645] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:26 2022] GET /identity/v3/role_assignments?user.id=83f4897f360f4f23aac5749685fdf500&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:27.165657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.171233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b7228-152b-42d6-b930-778b92990f27 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.184281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:27.225461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.226150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.226430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.226697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.227499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:27.227802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.229204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.241111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 325/646] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:27.241870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.269917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:27.299153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774ea99-eb73-4e21-b644-79aef6297bfb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.348377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.349341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.349808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.350236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.362621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.364118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.371716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5002fde-6bfa-4142-83aa-8507a0a965ee None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:27.385058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:27.386158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:27.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 322/647] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:27.387856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.410206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7df60-7102-4647-b13d-70d2ceec0e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.412327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7df60-7102-4647-b13d-70d2ceec0e89 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.412826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7df60-7102-4647-b13d-70d2ceec0e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.413097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f7df60-7102-4647-b13d-70d2ceec0e89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.425875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f7df60-7102-4647-b13d-70d2ceec0e89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-37f7df60-7102-4647-b13d-70d2ceec0e89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:27.431984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:27.432904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 326/648] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:27.434764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7df60-7102-4647-b13d-70d2ceec0e89 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.479918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:27.515778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241884b-1b70-4f79-b79f-7ff9347dd90c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.561420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.562368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.562646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.562916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.563635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f7df60-7102-4647-b13d-70d2ceec0e89 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.574121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:27.577087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.578922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:27.598626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:27.599777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:27.600853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:27.600853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:27.600853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:27.606074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 323/649] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:27.606074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.609056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.609909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.610324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.610734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.612050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:27.612641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.614506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.635622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 327/650] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:27.641363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.729865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459dadb4-5fdf-43fd-9475-2463999776f0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.734283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:27.747645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17eeff0-3afb-4a82-891b-8858ab133513 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.776949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.778587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.779361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.779830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.780874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:27.781423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.783083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.783350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.783833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.784168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.784711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.785482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:27.785870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.787349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.794876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 324/651] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:27.795955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.804301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 328/652] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:27.805382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:27.861821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:27.909604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef921a78-991b-499c-9a88-42f6976c4148 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.915911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789301e1-8959-4731-99c1-7c9ff13ae773 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:27.945607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c627ef4-3cdb-444f-b12d-5e6615fe7ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.946410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c627ef4-3cdb-444f-b12d-5e6615fe7ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.946745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c627ef4-3cdb-444f-b12d-5e6615fe7ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.947076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c627ef4-3cdb-444f-b12d-5e6615fe7ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.953848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:27.954488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:27.954766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:27.955043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:27.967661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:27.969109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:27.978632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:27.986936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:27.988055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:27.989206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:27.989206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:27.989206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:27.989206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 329/653] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:27.989206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.024480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c627ef4-3cdb-444f-b12d-5e6615fe7ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.026429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c627ef4-3cdb-444f-b12d-5e6615fe7ec7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.086225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:28.100736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ccd75-fa30-48bc-bd58-7d3b199383d4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:28.140465 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:28.141728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.142282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.142722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.151626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 325/654] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:28.151626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c627ef4-3cdb-444f-b12d-5e6615fe7ec7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.156762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.158639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:28.171140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:28.172259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 05 13:58:28.173374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:28.180337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 330/655] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:27 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:28.181476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.252454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c627ef4-3cdb-444f-b12d-5e6615fe7ec7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:28.266783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:28.291333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:28.292263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.292749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.293193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.294091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:28.294560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.296159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.310248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 326/656] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:28.311391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.319743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef0e1bb-aa4b-4a3e-8982-ba027ba8873b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:28.330668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:28.355928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:28.356897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.357329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.357791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.370972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.372610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:28.387402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:28.388542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:28.389545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:28.389545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:28.389545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:28.395875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 331/657] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:28.396937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.436457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73334f34-0ae6-42b9-a903-76f2674131b9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:28.483322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:28.484215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.484499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.484772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.485519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:28.486092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.488313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.508196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44246d2-be86-416a-bdee-5d49e3e153e6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:28.516529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 327/658] 10.222.0.23 () {56 vars in 1137 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:28.517933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.527139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:28.550865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:28.551597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.551914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.552184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.560729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.562237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:28.574967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:28.576557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 332/659] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:28.577694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.622553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:28.634461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b187a819-6c87-416a-9dbf-218888f18980 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:28.670186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:28.670806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.671087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.671361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.672005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:28.672318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.673875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.685932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 328/660] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:28.686882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.708248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a23107c-0b01-4b8d-8c7c-815763457fe1 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:28.756388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ae881-d40c-4b17-a1a3-1195def7160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:28.757562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae881-d40c-4b17-a1a3-1195def7160a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.758008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae881-d40c-4b17-a1a3-1195def7160a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.759071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae881-d40c-4b17-a1a3-1195def7160a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.794402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cc56c4-d281-4b85-8d3a-193892c8fbee None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:28.816330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:28.837780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:28.844002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:28.844635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.844911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.845184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.845938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:28.846246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.847761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.848167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ae881-d40c-4b17-a1a3-1195def7160a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:28.849734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ae881-d40c-4b17-a1a3-1195def7160a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:28.865863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 329/661] 10.222.0.23 () {56 vars in 1139 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:28.866721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.964140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 333/662] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:28.964140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c12c-56e1-4e52-a964-06485bab1e6d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:28.964140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c12c-56e1-4e52-a964-06485bab1e6d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:28.964140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a16c12c-56e1-4e52-a964-06485bab1e6d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:28.964140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 334/663] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:28.964140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16c12c-56e1-4e52-a964-06485bab1e6d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:28.978831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b261917e-68a0-4016-a66f-2bad42d80cd2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.015406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.015976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.016251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.016514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.023413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:29.024855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:29.036910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:29.037836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:29.038724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:29.038724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:29.038724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:29.038724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:29.038724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:29.038724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:29.038724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:29.038724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:29.044486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 330/664] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:29.044486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:29.046131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:29.073884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a16c12c-56e1-4e52-a964-06485bab1e6d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.105242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:29.109796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183124dd-50e5-4d4a-b2a3-5e6161a9c496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.110502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183124dd-50e5-4d4a-b2a3-5e6161a9c496 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.110847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183124dd-50e5-4d4a-b2a3-5e6161a9c496 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.111124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183124dd-50e5-4d4a-b2a3-5e6161a9c496 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.161193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312b868b-2752-4a17-ba75-de1f4a2809e9 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.198160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183124dd-50e5-4d4a-b2a3-5e6161a9c496 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:29.198414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f98c36e-05f2-4453-b5dd-c76a96acb959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.198414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98c36e-05f2-4453-b5dd-c76a96acb959 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.198414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98c36e-05f2-4453-b5dd-c76a96acb959 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.198640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183124dd-50e5-4d4a-b2a3-5e6161a9c496 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:29.198717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f98c36e-05f2-4453-b5dd-c76a96acb959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.211506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98c36e-05f2-4453-b5dd-c76a96acb959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:29.213325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f98c36e-05f2-4453-b5dd-c76a96acb959 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4f98c36e-05f2-4453-b5dd-c76a96acb959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:29.228005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:29.229173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:29.230100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:29.230100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:29.230100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:29.230100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 331/665] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:29.230474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bb5ac-a7c2-4136-84c0-804dc845c593 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.230759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bb5ac-a7c2-4136-84c0-804dc845c593 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.231027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4bb5ac-a7c2-4136-84c0-804dc845c593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.250689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0c4bb5ac-a7c2-4136-84c0-804dc845c593 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:29.251854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0c4bb5ac-a7c2-4136-84c0-804dc845c593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:29.277986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:29.317082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 335/666] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:29.318013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183124dd-50e5-4d4a-b2a3-5e6161a9c496 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:29.337792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 332/667] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:29.338947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bb5ac-a7c2-4136-84c0-804dc845c593 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:29.380206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:29.436747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183124dd-50e5-4d4a-b2a3-5e6161a9c496 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.473904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4bb5ac-a7c2-4136-84c0-804dc845c593 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.479348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.480172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.480774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.481366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.482784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:29.483474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:29.486536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:29.515931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 336/668] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:29.517036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:29.527443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.528688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.529122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.529530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.530361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:29.530804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:29.532363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:29.546283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 333/669] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:29.547369 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:29.559711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:29.632539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b37c3-dab0-43d0-a1f0-80da625b32b9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.640334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:29.649200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0299c-85f1-4921-8e11-98ca2e77563b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.666553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.667177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.667447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.667711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.678319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:29.679752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:29.681870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f0a93-e4af-42e2-ba0d-78adfab521a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.682481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0a93-e4af-42e2-ba0d-78adfab521a5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.682767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0a93-e4af-42e2-ba0d-78adfab521a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.683038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f0a93-e4af-42e2-ba0d-78adfab521a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.683757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f0a93-e4af-42e2-ba0d-78adfab521a5 None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=44fba9b1749b4172bc9479ca980fcf48) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:29.684062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f0a93-e4af-42e2-ba0d-78adfab521a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:29.685497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f0a93-e4af-42e2-ba0d-78adfab521a5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:29.691298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 334/670] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/role_assignments?user.id=44fba9b1749b4172bc9479ca980fcf48&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:29.692419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076cd61-e3a0-4d9f-a2ae-27320d41a13b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.692698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076cd61-e3a0-4d9f-a2ae-27320d41a13b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.692970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076cd61-e3a0-4d9f-a2ae-27320d41a13b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:29.693654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:29.694837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:29.696041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:29.696041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:29.696041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:29.696259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 335/671] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:29.696259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076cd61-e3a0-4d9f-a2ae-27320d41a13b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:29.699009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 337/672] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:29.699910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:29.798841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:29.819621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced8b964-ccac-4f6d-9149-b7f6ba242964 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.824498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076cd61-e3a0-4d9f-a2ae-27320d41a13b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:29.861649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138f45b2-ced3-496a-9180-a812d0591b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.862858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f45b2-ced3-496a-9180-a812d0591b69 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.863398 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f45b2-ced3-496a-9180-a812d0591b69 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.863815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138f45b2-ced3-496a-9180-a812d0591b69 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.869232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:29.869906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.870183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.870443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.876556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138f45b2-ced3-496a-9180-a812d0591b69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:29.877865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:29.879428 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-138f45b2-ced3-496a-9180-a812d0591b69 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:29.881995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:29.883022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:29.883848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 338/673] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:29.884526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7609b2-3199-4c98-9743-ce3971c244e4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:29.885071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7609b2-3199-4c98-9743-ce3971c244e4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:29.885071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7609b2-3199-4c98-9743-ce3971c244e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:29.893601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:29.894738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:29.896442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:29.896442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:29.896442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:29.896442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:29.896442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:29.896442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:29.896442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:29.896442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:29.898121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 336/674] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:29.898932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:29.905061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4e7609b2-3199-4c98-9743-ce3971c244e4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:29.906300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4e7609b2-3199-4c98-9743-ce3971c244e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:29.919352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:29.982262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 339/675] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:29.983112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7609b2-3199-4c98-9743-ce3971c244e4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.016017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28233b98-3a78-463a-bab3-aa045bf7d59e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.060208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.061323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.061763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.062409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.064108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:30.064557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.066281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.068652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:30.084980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 337/676] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:30.086627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.109471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7609b2-3199-4c98-9743-ce3971c244e4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.153377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.154050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.154321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.154585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.155350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:30.155761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.157248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.172133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 340/677] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:30.173272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.180688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:30.207423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b42429-50d6-43d4-b2bf-b3bcc9fb60ff None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.244224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.244846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.245121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.245389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.246190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:30.246497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.248148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.261178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 338/678] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:30.262154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.279342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c7426-a9e7-44f4-baa8-72f536491a78 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.317725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.318639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.318998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.319361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.324653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:30.332574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.334562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:30.345995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:30.347189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:30.348217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 341/679] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:30.348801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.373806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d6fe64-29ba-40ba-810d-41148bfb63d5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.410417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.411209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.411586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.411959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.414628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:30.423159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.424633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:30.438906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:30.439903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:30.440814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:30.440814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:30.440814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:30.440814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 339/680] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:30.441550 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.459606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ce98e-b67e-4d24-b2b4-2c9f3f3d49ff None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.495089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.495705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.496138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.496407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.497426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=417b661f770846f189c36d0804b6dce5) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:30.497755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.499195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.511047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 342/681] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/role_assignments?user.id=417b661f770846f189c36d0804b6dce5&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:30.512351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.561878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea9242-abad-4ef2-b535-60eb261d0d3d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.597118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:30.610068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc8834-b894-4cd1-9802-bea1faec700f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.610721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8834-b894-4cd1-9802-bea1faec700f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.610994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8834-b894-4cd1-9802-bea1faec700f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.611613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8834-b894-4cd1-9802-bea1faec700f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.624323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc8834-b894-4cd1-9802-bea1faec700f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1bcc8834-b894-4cd1-9802-bea1faec700f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:30.630854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:30.631953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:30.632784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:30.637387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 340/682] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:30.638504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc8834-b894-4cd1-9802-bea1faec700f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.646907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab13209-5f89-47de-a22e-3b1cdc23763f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.683518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.684140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.684417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.684674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.685369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:30.685691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.687135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.692940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:30.705714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 343/683] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:30.705714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.758704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc8834-b894-4cd1-9802-bea1faec700f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.796241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.796848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.797122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.797388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.798064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:30.798364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.800355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.812481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 341/684] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:30.812643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.812820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58a11c4-50ca-4de6-a476-68d467135599 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.836395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:30.855823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.856477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.856761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.857059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.858430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:30.858430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.860591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.869794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 344/685] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:30.871018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.924562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e329c-70a9-4b6f-b72d-1037ddeeef97 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:30.935542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:30.964648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:30.965732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:30.966011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:30.966278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:30.967353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=6c9b940cd9e0432daa235f82becb4d38) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:30.967664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:30.969694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:30.977288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 342/686] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/role_assignments?user.id=6c9b940cd9e0432daa235f82becb4d38&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:30.978579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:30.991590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a178cb5-a9d0-4e00-8212-d01fe460f272 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:31.031377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:31.032057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:31.032337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:31.032608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:31.042264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:31.043911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:31.056799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:31.057885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:31.059334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 345/687] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:31.066534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:31.103269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73aec90-92c3-4dd4-99d5-17a6ba3a1f96 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:31.118507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:31.146361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:31.147007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:31.147279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:31.147548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:31.161869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:31.163714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:31.199110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:31.200157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:31.201053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:31.201053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:31.201053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:31.201053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 343/688] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:31.206928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:31.212030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4640385-d983-4ebb-befa-5322922ae1b8 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:31.253061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:31.285575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:31.286646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:31.287238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:31.287706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:31.288661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:31.289163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:31.291605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:31.318564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 346/689] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 13:58:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:31.324551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:31.351046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddda1a7-03cd-46b3-86ef-4aeacfdc943b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:31.395392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:31.396054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:31.396395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:31.396731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:31.398138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:31.398299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:31.400493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:31.417120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 344/690] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:31.425545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:31.448959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:31.452932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621dfd0-c853-418e-88c8-adeb2e84a138 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:31.495829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200bf0d-8548-4347-8ef4-75a858a4d87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:31.496789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200bf0d-8548-4347-8ef4-75a858a4d87e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:31.497241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200bf0d-8548-4347-8ef4-75a858a4d87e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:31.497963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1200bf0d-8548-4347-8ef4-75a858a4d87e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:31.529431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:31.547152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d05d88f-6275-46ff-b123-24b1fd20cf8f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:31.585004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:31.585690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:31.586175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:31.587082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:31.589059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200bf0d-8548-4347-8ef4-75a858a4d87e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:31.591473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1200bf0d-8548-4347-8ef4-75a858a4d87e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:31.597088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:31.598704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:31.614062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:31.615123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:31.616080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 345/691] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:58:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:31.620984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:31.717982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:31.723908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 347/692] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 05 13:58:31.732708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200bf0d-8548-4347-8ef4-75a858a4d87e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:31.799674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d99656-cf53-4b35-af5e-11f488218241 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:31.843479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1200bf0d-8548-4347-8ef4-75a858a4d87e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:31.855720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:31.856347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:31.856622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:31.856891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:31.857643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:31.857962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:31.859419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:31.881750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 346/693] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:58:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:31.883225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:31.890919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:31.891585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:31.892052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:31.892304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:31.892584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:31.918521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:31.919989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:31.956710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:31.957965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:31.959049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:31.959049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:31.959049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:31.959049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 348/694] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:31.963075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:31.992207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ccd11-ccf5-42a0-bdb5-2e32c620b834 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:32.028870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fef61c-8cab-4ae7-9c69-1054422f4ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:32.029851 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fef61c-8cab-4ae7-9c69-1054422f4ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.030300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fef61c-8cab-4ae7-9c69-1054422f4ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.031209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fef61c-8cab-4ae7-9c69-1054422f4ff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.101040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:32.125668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fef61c-8cab-4ae7-9c69-1054422f4ff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:32.127781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fef61c-8cab-4ae7-9c69-1054422f4ff6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:32.144853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:32.189447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddc16f-2cbe-4bd8-9711-ff9cc1dac446 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:32.233713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:32.234841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.235384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.235824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.236983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:32.237533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:32.239980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:32.248719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 347/695] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:32.248719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3824-45d0-4a2c-a580-d12be467d581 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.248719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3824-45d0-4a2c-a580-d12be467d581 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.248719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78d3824-45d0-4a2c-a580-d12be467d581 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.248719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 348/696] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:32.255055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf6d7a-77c1-456b-9023-95fe1cb241c8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.255342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf6d7a-77c1-456b-9023-95fe1cb241c8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.255616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf6d7a-77c1-456b-9023-95fe1cb241c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.259541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 349/697] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:32.260593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:32.279970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2fcf6d7a-77c1-456b-9023-95fe1cb241c8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:32.281252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2fcf6d7a-77c1-456b-9023-95fe1cb241c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:32.370066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 349/698] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:32.371067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf6d7a-77c1-456b-9023-95fe1cb241c8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:32.371869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898e663a-cea1-499d-b667-70c43887d3d6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:32.386109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:32.411174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:32.412099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.412514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.412917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.424472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:32.426214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:32.426672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:32.442828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:32.443981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:32.445084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:32.445084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:32.445084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:32.451269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 350/699] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:32.452853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:32.567179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea72f21-ae3a-470c-95da-ce5afe9a4a94 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:32.569452 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf6d7a-77c1-456b-9023-95fe1cb241c8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:32.618192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:32.618864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.619147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.619415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.620066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:32.620364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:32.621752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:32.636633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:32.637764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.638048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.638466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.641498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 351/700] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:32.649808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:32.651338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:32.654399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:32.665830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:32.666748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:32.667654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 350/701] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:32.674307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:32.706008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:32.821285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68990189-cb8f-4783-8799-7c117e96192a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:32.823510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:32.833884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7cdf17-7341-40b9-8e5d-6f9cde64000f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:32.859251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:32.860005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.860315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.860684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.869827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:32.871134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:32.871411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:32.872186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:32.872676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:32.873229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:32.874248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:32.874751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:32.876391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:32.885197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:32.886173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:32.887051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:32.887051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:32.887051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:32.887051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:32.887051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:32.887051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:32.887051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:32.887051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:32.889714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 352/702] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:32.890820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:32.901780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 351/703] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:32.901780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:32.986657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:33.008190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e7a1d-53b3-46e7-a33f-6a0df60bdc6b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:33.015716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c8671-de70-4264-b791-130264ee5b47 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:33.049181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:33.050323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.050817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.051264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.052030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:33.053599 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:33.054132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:33.054813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.055157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.055509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.056263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:33.056633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:33.058172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:33.067011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:33.068063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:33.068897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:33.076200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 352/705] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:33.076381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 353/705] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Oct 5 13:58:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:33.077109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:33.077532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:33.093274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:33.184068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05c87e8-db5a-4fb7-ad86-44a5d8ebdd9a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:33.194666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7005c6-61f5-4acd-8b9c-4c36139505ec None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:33.218224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:33.218830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.219105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.219549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.220089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=01286f6f65b44519b645927f6c70ec5b) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:33.220399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:33.222027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:33.237666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 353/706] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:33 2022] GET /identity/v3/role_assignments?user.id=01286f6f65b44519b645927f6c70ec5b&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:33.237666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:33.239959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a8eb8f-a956-4780-8fcf-9adf99e4e218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:33.240673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8eb8f-a956-4780-8fcf-9adf99e4e218 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.241064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8eb8f-a956-4780-8fcf-9adf99e4e218 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.241402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a8eb8f-a956-4780-8fcf-9adf99e4e218 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.242200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8eb8f-a956-4780-8fcf-9adf99e4e218 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:33.242589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8eb8f-a956-4780-8fcf-9adf99e4e218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:33.244163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a8eb8f-a956-4780-8fcf-9adf99e4e218 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:33.249757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 354/707] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 13:58:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:33.255447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01855-9756-4432-af86-ab1eaea3fe17 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.255762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01855-9756-4432-af86-ab1eaea3fe17 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.256263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a01855-9756-4432-af86-ab1eaea3fe17 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.263832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 355/708] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:33.263832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01855-9756-4432-af86-ab1eaea3fe17 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:33.279524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:33.349332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:33.376762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71dd4f-e565-43bb-9cbb-2f7819bc0a82 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:33.410640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a01855-9756-4432-af86-ab1eaea3fe17 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:33.415388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4a4931-a5b3-4d9e-ace5-5b829b527718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:33.416166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4931-a5b3-4d9e-ace5-5b829b527718 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.416530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4931-a5b3-4d9e-ace5-5b829b527718 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.416886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4a4931-a5b3-4d9e-ace5-5b829b527718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.417769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a4931-a5b3-4d9e-ace5-5b829b527718 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=83f4897f360f4f23aac5749685fdf500) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:33.418166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a4931-a5b3-4d9e-ace5-5b829b527718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:33.419790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4a4931-a5b3-4d9e-ace5-5b829b527718 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:33.434040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 354/709] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:33 2022] GET /identity/v3/role_assignments?user.id=83f4897f360f4f23aac5749685fdf500&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:33.435776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf854f-d358-4c77-a33e-e53fb2a7275d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.436061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf854f-d358-4c77-a33e-e53fb2a7275d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.436334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf854f-d358-4c77-a33e-e53fb2a7275d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.454379 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065817e8-ac86-469a-8507-c190fde3c588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:33.455026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065817e8-ac86-469a-8507-c190fde3c588 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.455310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065817e8-ac86-469a-8507-c190fde3c588 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.455584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065817e8-ac86-469a-8507-c190fde3c588 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.456310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065817e8-ac86-469a-8507-c190fde3c588 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:33.456615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065817e8-ac86-469a-8507-c190fde3c588 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:33.458022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-59cf854f-d358-4c77-a33e-e53fb2a7275d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:33.459074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065817e8-ac86-469a-8507-c190fde3c588 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:33.459299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-59cf854f-d358-4c77-a33e-e53fb2a7275d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:33.468798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 356/710] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 13:58:33 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:33.559841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 355/711] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:33.702198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25521-c6b6-498b-9b5f-85341ed91e75 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.702476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25521-c6b6-498b-9b5f-85341ed91e75 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.702744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb25521-c6b6-498b-9b5f-85341ed91e75 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.704249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 357/712] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:33.710168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf5d1-f6c5-499f-82fe-32a463dc3b11 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:33.710626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf5d1-f6c5-499f-82fe-32a463dc3b11 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:33.710722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdf5d1-f6c5-499f-82fe-32a463dc3b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:33.731192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1bbdf5d1-f6c5-499f-82fe-32a463dc3b11 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:33.732376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1bbdf5d1-f6c5-499f-82fe-32a463dc3b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:33.811321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:33.817828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 356/713] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:33.849252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb25521-c6b6-498b-9b5f-85341ed91e75 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:33.928563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:33.956963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb25521-c6b6-498b-9b5f-85341ed91e75 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:34.001046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:34.001710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:34.001984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:34.002247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:34.019722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:34.021223 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:34.041305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:34.042410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:34.044610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:34.044610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:34.044610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:34.044610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 358/714] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:34.048386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdf5d1-f6c5-499f-82fe-32a463dc3b11 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:34.120093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:34.189053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdf5d1-f6c5-499f-82fe-32a463dc3b11 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:34.225854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:34.226380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:34.226658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:34.226929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:34.228067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:34.228373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:34.233657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:34.243597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 357/715] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:34.248950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:34.322039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:34.358448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a3495a-6a45-410c-b035-a95f627632f1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:34.400186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e86254-4b17-448e-a219-a0a30d0c9a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:34.400814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e86254-4b17-448e-a219-a0a30d0c9a83 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:34.401092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e86254-4b17-448e-a219-a0a30d0c9a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:34.401795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e86254-4b17-448e-a219-a0a30d0c9a83 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:34.402457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e86254-4b17-448e-a219-a0a30d0c9a83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:34.404211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e86254-4b17-448e-a219-a0a30d0c9a83 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:34.405872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796bbb67-ea28-4643-a492-1b5c011e926a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:34.431692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 358/716] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:58:34 2022] POST /identity/v3/users => generated 254 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:34.470741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:34.482718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:34.483351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:34.483633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:34.483900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] PATH_INFO: `/v3/services/network` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:34.484572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:34.486031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:34.496382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:34.497433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:34.498341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 359/717] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:34 2022] GET /identity/v3/services/network => generated 88 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:34.511115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e86254-4b17-448e-a219-a0a30d0c9a83 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:34.649805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e86254-4b17-448e-a219-a0a30d0c9a83 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:34.671084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:34.695337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984f29f-0fda-4115-8fbb-0655e0ad9f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:34.695973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984f29f-0fda-4115-8fbb-0655e0ad9f0c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:34.696248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984f29f-0fda-4115-8fbb-0655e0ad9f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:34.696516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984f29f-0fda-4115-8fbb-0655e0ad9f0c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:34.697163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984f29f-0fda-4115-8fbb-0655e0ad9f0c None admin] RBAC: Adding query filter params (name=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:34.697464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984f29f-0fda-4115-8fbb-0655e0ad9f0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:34.698896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984f29f-0fda-4115-8fbb-0655e0ad9f0c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:34.709974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 359/718] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:58:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:34.715827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:34.825933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3869b2c-c9a0-4b5d-8c27-e55934533012 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:34.865421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f308a1-e9ea-40a3-bbe1-bd11f312cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:34.866879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f308a1-e9ea-40a3-bbe1-bd11f312cb87 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:34.867346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f308a1-e9ea-40a3-bbe1-bd11f312cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:34.867841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f308a1-e9ea-40a3-bbe1-bd11f312cb87 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:34.868690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f308a1-e9ea-40a3-bbe1-bd11f312cb87 None admin] RBAC: Adding query filter params (type=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:34.869190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f308a1-e9ea-40a3-bbe1-bd11f312cb87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:34.870770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f308a1-e9ea-40a3-bbe1-bd11f312cb87 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:34.876111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 360/719] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:58:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:36.015381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62279d8-799e-4308-ac51-4318646a8029 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.016444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62279d8-799e-4308-ac51-4318646a8029 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.016928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62279d8-799e-4308-ac51-4318646a8029 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.018849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 360/720] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:36.025234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2fbe5-c2c2-4176-a58b-e569632b7711 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.026240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2fbe5-c2c2-4176-a58b-e569632b7711 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.026736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f2fbe5-c2c2-4176-a58b-e569632b7711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.052059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d6f2fbe5-c2c2-4176-a58b-e569632b7711 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:36.053280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d6f2fbe5-c2c2-4176-a58b-e569632b7711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:36.054032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153f3d6-7a62-469b-857e-f5e9ee579af3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.054514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153f3d6-7a62-469b-857e-f5e9ee579af3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.054998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4153f3d6-7a62-469b-857e-f5e9ee579af3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.057921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 361/721] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:36.063477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6afcd-efa6-4619-9e24-311239a2b873 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.063755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6afcd-efa6-4619-9e24-311239a2b873 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.064019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6afcd-efa6-4619-9e24-311239a2b873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.094103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-80f6afcd-efa6-4619-9e24-311239a2b873 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:36.096084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-80f6afcd-efa6-4619-9e24-311239a2b873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:36.135299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:36.173028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 361/722] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 05 13:58:36.227781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 362/723] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:36.247297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6853d-1837-4249-988c-6e161b2bcb96 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.247883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6853d-1837-4249-988c-6e161b2bcb96 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.248375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b6853d-1837-4249-988c-6e161b2bcb96 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.250162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 362/724] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:36.255553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcb72-7fb1-4458-a45a-9c5de0fbfee5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.256063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcb72-7fb1-4458-a45a-9c5de0fbfee5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.256540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75dcb72-7fb1-4458-a45a-9c5de0fbfee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.286751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d75dcb72-7fb1-4458-a45a-9c5de0fbfee5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:36.287852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d75dcb72-7fb1-4458-a45a-9c5de0fbfee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:36.373222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 363/725] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:36.570669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6853d-1837-4249-988c-6e161b2bcb96 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:36.626600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dcb72-7fb1-4458-a45a-9c5de0fbfee5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:36.673485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b6853d-1837-4249-988c-6e161b2bcb96 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:36.691372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:36.712258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee029af5-043b-4877-b7cf-d97c83f195e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:36.713247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee029af5-043b-4877-b7cf-d97c83f195e5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.713756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee029af5-043b-4877-b7cf-d97c83f195e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.714213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee029af5-043b-4877-b7cf-d97c83f195e5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.728427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee029af5-043b-4877-b7cf-d97c83f195e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ee029af5-043b-4877-b7cf-d97c83f195e5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:36.733471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:36.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:36.735200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:36.738175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 363/726] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:36.743716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee029af5-043b-4877-b7cf-d97c83f195e5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:36.764724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75dcb72-7fb1-4458-a45a-9c5de0fbfee5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:36.796844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:36.798846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18b546-46cf-4e93-a40d-11bb949b75bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:36.799780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18b546-46cf-4e93-a40d-11bb949b75bb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.800070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18b546-46cf-4e93-a40d-11bb949b75bb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.800348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d18b546-46cf-4e93-a40d-11bb949b75bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.810810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d18b546-46cf-4e93-a40d-11bb949b75bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8d18b546-46cf-4e93-a40d-11bb949b75bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:36.815652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:36.816589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:36.817419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:36.821995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 364/727] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:36.822676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18b546-46cf-4e93-a40d-11bb949b75bb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:36.864679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee029af5-043b-4877-b7cf-d97c83f195e5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:36.907158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047977a4-b885-44b6-8b57-27f3ceb8b771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:36.908515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047977a4-b885-44b6-8b57-27f3ceb8b771 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:36.909300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047977a4-b885-44b6-8b57-27f3ceb8b771 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:36.909809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047977a4-b885-44b6-8b57-27f3ceb8b771 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:36.911035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047977a4-b885-44b6-8b57-27f3ceb8b771 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:36.912619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047977a4-b885-44b6-8b57-27f3ceb8b771 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:36.923418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:36.928020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d18b546-46cf-4e93-a40d-11bb949b75bb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:36.940785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 364/728] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:58:36 2022] POST /identity/v3/services => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:36.942295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047977a4-b885-44b6-8b57-27f3ceb8b771 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.018714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.019269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.019547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.019704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.020371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:37.020671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.022116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.044177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 365/729] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:37.061814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.079468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047977a4-b885-44b6-8b57-27f3ceb8b771 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.088595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:37.125742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.126805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.127440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.127977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.128893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:37.129408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.131462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.143167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 365/730] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:37.149729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.261219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7035b66-2e67-454b-a37b-f4c6c83522ec None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.279704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9930a2-c66e-4723-8852-291188fcbd0d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.301466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.302263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.302626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.303032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.315524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.317363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.318172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b4c23-4500-471e-a00b-4f48dfa515af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.318998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b4c23-4500-471e-a00b-4f48dfa515af None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.319688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b4c23-4500-471e-a00b-4f48dfa515af None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.320110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b4c23-4500-471e-a00b-4f48dfa515af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.333482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b4c23-4500-471e-a00b-4f48dfa515af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:37.334502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:37.335595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:37.336690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:37.336690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:37.336690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:37.337816 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b4c23-4500-471e-a00b-4f48dfa515af None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.338192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 366/731] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:37.344331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ca5b4c23-4500-471e-a00b-4f48dfa515af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:37.355517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:37.356621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:37.357475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:37.357475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:37.357475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:37.365546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 366/732] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:37.365546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938462a-601c-44c1-8de5-d126456c2957 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.365824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:37.365902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938462a-601c-44c1-8de5-d126456c2957 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.365902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c938462a-601c-44c1-8de5-d126456c2957 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.365902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 367/733] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:37.369934 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938462a-601c-44c1-8de5-d126456c2957 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.397481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:37.469333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e1d2f0-0964-4baa-b07f-bcc55050b288 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.491032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c938462a-601c-44c1-8de5-d126456c2957 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.509474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9368f5-5b94-4230-89eb-7e6ca20ec501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.510138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9368f5-5b94-4230-89eb-7e6ca20ec501 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.510409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9368f5-5b94-4230-89eb-7e6ca20ec501 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.510671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9368f5-5b94-4230-89eb-7e6ca20ec501 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.511316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9368f5-5b94-4230-89eb-7e6ca20ec501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:37.511613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9368f5-5b94-4230-89eb-7e6ca20ec501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.513013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9368f5-5b94-4230-89eb-7e6ca20ec501 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.526175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 367/734] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:37.527557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73321f78-b6d5-4f7c-9638-e45b5304cc7b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.527837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73321f78-b6d5-4f7c-9638-e45b5304cc7b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.528102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73321f78-b6d5-4f7c-9638-e45b5304cc7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.533957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b7b3a-2fda-4b1d-b5f8-e2914b900f88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.534869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7b3a-2fda-4b1d-b5f8-e2914b900f88 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.535297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7b3a-2fda-4b1d-b5f8-e2914b900f88 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.535568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b7b3a-2fda-4b1d-b5f8-e2914b900f88 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.536259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b7b3a-2fda-4b1d-b5f8-e2914b900f88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:37.536558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b7b3a-2fda-4b1d-b5f8-e2914b900f88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.537974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b7b3a-2fda-4b1d-b5f8-e2914b900f88 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.546429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-73321f78-b6d5-4f7c-9638-e45b5304cc7b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:37.548040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73321f78-b6d5-4f7c-9638-e45b5304cc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:37.550166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 368/735] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:37.551561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d6508-4fa9-4f17-9925-2bba13421d26 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.551835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d6508-4fa9-4f17-9925-2bba13421d26 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.552105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d6508-4fa9-4f17-9925-2bba13421d26 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.553435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 369/736] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:37.554661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d6508-4fa9-4f17-9925-2bba13421d26 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.615459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:37.620757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 368/737] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:37.621999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73321f78-b6d5-4f7c-9638-e45b5304cc7b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.651481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:37.675205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d6508-4fa9-4f17-9925-2bba13421d26 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.710974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239ae0f-7d95-4751-b7a0-49c3b208523d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.711699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ae0f-7d95-4751-b7a0-49c3b208523d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.712044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ae0f-7d95-4751-b7a0-49c3b208523d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.712402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239ae0f-7d95-4751-b7a0-49c3b208523d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.719766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239ae0f-7d95-4751-b7a0-49c3b208523d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.721296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239ae0f-7d95-4751-b7a0-49c3b208523d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f239ae0f-7d95-4751-b7a0-49c3b208523d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:37.732995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:37.734087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:37.735341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 370/738] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:37.735944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbf70b-800c-4d75-a3c6-277f65ef4842 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.736467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbf70b-800c-4d75-a3c6-277f65ef4842 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.737057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dbf70b-800c-4d75-a3c6-277f65ef4842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.740822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73321f78-b6d5-4f7c-9638-e45b5304cc7b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.755457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-45dbf70b-800c-4d75-a3c6-277f65ef4842 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:37.756769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-45dbf70b-800c-4d75-a3c6-277f65ef4842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:37.777293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.778310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.778840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.779318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.787487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.789428 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:37.802218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:37.803152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:37.804101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:37.806808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 369/739] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:37.807936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.823330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 371/740] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 05 13:58:37.824200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbf70b-800c-4d75-a3c6-277f65ef4842 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.872557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:37.898013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:37.931750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dbf70b-800c-4d75-a3c6-277f65ef4842 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.936044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecef6c0-19e6-4935-9346-2af034513ed2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:37.968391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.969050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.969330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.969600 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.970371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:37.970671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.972151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:37.986635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:37.988175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:37.988627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:37.989297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:37.990249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:37.990749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:37.991330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 372/741] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:37.992166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:37.992948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.017150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 370/742] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:38.018077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.120149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c784aa50-6180-4db6-9208-7630ff0df768 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.125746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a821664-88a8-47a6-a498-8dc54835aa7e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.150481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:38.152528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:38.166010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009e4a7-4a46-475d-a8cb-fdf49cc24f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.166942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009e4a7-4a46-475d-a8cb-fdf49cc24f1e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.167362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009e4a7-4a46-475d-a8cb-fdf49cc24f1e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.167814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009e4a7-4a46-475d-a8cb-fdf49cc24f1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.171048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db9d00-bfb9-45ca-aedb-4df387774f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.171666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db9d00-bfb9-45ca-aedb-4df387774f11 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.171945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db9d00-bfb9-45ca-aedb-4df387774f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.172219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db9d00-bfb9-45ca-aedb-4df387774f11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.179366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009e4a7-4a46-475d-a8cb-fdf49cc24f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3009e4a7-4a46-475d-a8cb-fdf49cc24f1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:38.184324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:38.185446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:38.186549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:38.188856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 373/743] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:38.189848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009e4a7-4a46-475d-a8cb-fdf49cc24f1e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.245685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db9d00-bfb9-45ca-aedb-4df387774f11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:38.247266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db9d00-bfb9-45ca-aedb-4df387774f11 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.318675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009e4a7-4a46-475d-a8cb-fdf49cc24f1e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.353199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.354116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.354116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.354388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.357913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 371/744] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 05 13:58:38.358904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db9d00-bfb9-45ca-aedb-4df387774f11 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.367168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:38.368661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.382904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:38.387619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:38.388541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:38.389827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:38.389827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:38.389827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:38.389827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 374/745] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:38.390646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.436790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:38.478960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db9d00-bfb9-45ca-aedb-4df387774f11 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.499325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb2d4c-3d87-46db-96a9-b2a5886a3c0d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.520812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c09b-0768-4e0b-a5de-497f48805d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.521538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c09b-0768-4e0b-a5de-497f48805d46 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.521912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c09b-0768-4e0b-a5de-497f48805d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.522186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61c09b-0768-4e0b-a5de-497f48805d46 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.522873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61c09b-0768-4e0b-a5de-497f48805d46 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:38.524254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d61c09b-0768-4e0b-a5de-497f48805d46 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.535948 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.536520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.536809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.537156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.537793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:38.545759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:38.546047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 372/746] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:58:38 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:38.546047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c09b-0768-4e0b-a5de-497f48805d46 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.546214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.554104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 375/747] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:38.555006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.646214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:38.676232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4675e28a-ac21-4254-a3ee-9c0bb3008ddd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.679547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:38.685381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d61c09b-0768-4e0b-a5de-497f48805d46 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.718884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.719513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.719961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.720232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.721221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:38.721892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:38.723275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.725969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.726726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.727043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.727372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.736744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 376/748] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:38.737605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.751011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:38.752526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:38.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:38.771207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:38.772090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:38.772090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:38.772090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:38.772090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 373/749] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:38.772090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.863837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9d8812-96d9-4262-bde0-288d8e30acdd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.882737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15ff32-6a55-4ea7-b896-e4d9c9bfb0c1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:38.904745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.905392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.905941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.906227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.910561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:38.921135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:38.921135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.925315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:38.926259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:38.926682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:38.927117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:38.935373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:38.937034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:38.938566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:38.939640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:38.940551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:38.940551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:38.940551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:38.941134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 377/750] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:38.942572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:38.953568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:38.954685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:38.955558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:38.955558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:38.955558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:38.955558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:38.955558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:38.955558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:38.955558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:38.955558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:38.958053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 374/751] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:38.958989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:38.962677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:39.066470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfaaf65-5252-4fe6-9e33-bca3a7792d01 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.067892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d1f4e-b8ab-4e9e-ba59-70da2edb6f2a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.102451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.103435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.103955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.104424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.105270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:39.105803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.107213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.107651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.107824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.108102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.108372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.109020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:39.109318 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.110713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.122238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 375/752] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:39.122888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 378/753] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:39.123950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:39.134646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:39.175290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:39.226747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:39.229286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3128d0a0-02d0-427a-b175-c65da8663b0a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.255322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c866f-ff89-48bf-bf3b-e22dded0a08f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.267313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.267993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.268602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.268720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.269398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:39.269817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.271183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.289697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 376/754] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:39.289697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:39.308077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.309157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.309765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.310227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.319115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.320935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:39.335165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:39.336160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:39.337085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:39.337085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:39.337085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:39.337085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:39.337085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:39.337085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:39.337085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:39.337085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:39.344307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 379/755] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:39.344307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:39.428568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153896a0-c0e5-4b5b-9c8f-c67384d63d4d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.437903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:39.460197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ad6a1-93db-4287-b8aa-6c0d4492fd64 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.472906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.473913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.474516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.475328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.484619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.487572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.497459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe5a59-c0e1-4898-ba1a-afbea8ecd848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.498193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe5a59-c0e1-4898-ba1a-afbea8ecd848 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.498474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe5a59-c0e1-4898-ba1a-afbea8ecd848 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.498822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe5a59-c0e1-4898-ba1a-afbea8ecd848 None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/83f4897f360f4f23aac5749685fdf500/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.500915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe5a59-c0e1-4898-ba1a-afbea8ecd848 None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=83f4897f360f4f23aac5749685fdf500, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.502457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe5a59-c0e1-4898-ba1a-afbea8ecd848 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:39.504208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:39.505127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:58:39.506124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:39.506651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:39.512168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 377/756] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 13:58:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:39.512168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:39.524580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 380/757] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:58:39 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/83f4897f360f4f23aac5749685fdf500/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:39.526177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a44996-6142-4a83-8b04-22b2ae6c8190 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.526579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a44996-6142-4a83-8b04-22b2ae6c8190 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.527072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a44996-6142-4a83-8b04-22b2ae6c8190 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.528791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 381/758] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:39.529745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a44996-6142-4a83-8b04-22b2ae6c8190 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:39.639033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c78ebf-0a81-48d0-acdd-c5d3507a8817 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.694389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2557cbc9-ed4b-4ca5-b236-2e382d1f217e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.694974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557cbc9-ed4b-4ca5-b236-2e382d1f217e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.695253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557cbc9-ed4b-4ca5-b236-2e382d1f217e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.695529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2557cbc9-ed4b-4ca5-b236-2e382d1f217e None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.696261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557cbc9-ed4b-4ca5-b236-2e382d1f217e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:39.696565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557cbc9-ed4b-4ca5-b236-2e382d1f217e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.698121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2557cbc9-ed4b-4ca5-b236-2e382d1f217e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.710580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a44996-6142-4a83-8b04-22b2ae6c8190 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.712044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 378/759] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:39.713420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa696bf1-3493-4e1e-848c-1c3e90894141 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.713734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa696bf1-3493-4e1e-848c-1c3e90894141 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.714003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa696bf1-3493-4e1e-848c-1c3e90894141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.734217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-aa696bf1-3493-4e1e-848c-1c3e90894141 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:39.735070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aa696bf1-3493-4e1e-848c-1c3e90894141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:39.755839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.756529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.756810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.757083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.757954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:39.758260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.759771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.760297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:39.781776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 382/760] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 13:58:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:39.784787 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:39.812066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 379/761] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:39.812929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa696bf1-3493-4e1e-848c-1c3e90894141 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:39.849858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:39.904060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dde7bd-7c8c-457d-841b-097db6bbdcea None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:39.949579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30766329-4912-4e06-9b77-f06550352897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:39.950624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766329-4912-4e06-9b77-f06550352897 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:39.951098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766329-4912-4e06-9b77-f06550352897 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:39.951557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30766329-4912-4e06-9b77-f06550352897 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:39.952586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30766329-4912-4e06-9b77-f06550352897 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=01286f6f65b44519b645927f6c70ec5b) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:39.953109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30766329-4912-4e06-9b77-f06550352897 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:39.954804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30766329-4912-4e06-9b77-f06550352897 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:39.967582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 383/762] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:39 2022] GET /identity/v3/role_assignments?user.id=01286f6f65b44519b645927f6c70ec5b&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:40.142873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30766329-4912-4e06-9b77-f06550352897 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:40.146865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa696bf1-3493-4e1e-848c-1c3e90894141 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:40.191615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655dea-24b3-4e05-9728-f90c79a1d12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:40.193363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655dea-24b3-4e05-9728-f90c79a1d12a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:40.193889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655dea-24b3-4e05-9728-f90c79a1d12a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:40.194484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a655dea-24b3-4e05-9728-f90c79a1d12a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:40.226377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:40.282642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655dea-24b3-4e05-9728-f90c79a1d12a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:40.284491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a655dea-24b3-4e05-9728-f90c79a1d12a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:40.362606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30766329-4912-4e06-9b77-f06550352897 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:40.387941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:40.432235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:40.432900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:40.433175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:40.433447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:40.434086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:40.435477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:40.451632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:40.452648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:40.453494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 384/763] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Oct 5 13:58:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:40.460069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:40.463396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 380/764] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:58:40.469347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655dea-24b3-4e05-9728-f90c79a1d12a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:40.590794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:40.606619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a655dea-24b3-4e05-9728-f90c79a1d12a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:40.662745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:40.664892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:40.665411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:40.666700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:40.681344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:40.683371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:40.688150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f7b2f-fa5f-444a-8ef6-979f53ee5d16 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:40.715060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:40.716100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:40.717000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:40.717000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:40.717000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:40.723821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 381/765] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:40.729398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:40.763201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:40.766087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:40.768358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:40.769420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:40.770973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:40.772718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:40.777989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:40.789164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 385/766] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 13:58:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:40.795132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:40.874907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6a22e2-148a-4716-8a03-22723dc2c1ec None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:40.885651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:40.900743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:40.919284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9344001-5e37-4135-a0c3-1ab983b12a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:40.919921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9344001-5e37-4135-a0c3-1ab983b12a55 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:40.920198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9344001-5e37-4135-a0c3-1ab983b12a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:40.920466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9344001-5e37-4135-a0c3-1ab983b12a55 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:40.921386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9344001-5e37-4135-a0c3-1ab983b12a55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:40.921709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9344001-5e37-4135-a0c3-1ab983b12a55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:40.923379 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9344001-5e37-4135-a0c3-1ab983b12a55 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:40.940725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 382/767] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:40.942442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1e2cf-6727-4b83-af70-92e6d055c84d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:40.942911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1e2cf-6727-4b83-af70-92e6d055c84d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:40.943363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c1e2cf-6727-4b83-af70-92e6d055c84d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:40.945016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 383/768] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:40.946803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1e2cf-6727-4b83-af70-92e6d055c84d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:40.974430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776991b3-265c-4747-8246-c57f454a0723 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:41.010475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42e5749-4929-4a0a-9d5a-e900ecd3ef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:41.011104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5749-4929-4a0a-9d5a-e900ecd3ef6c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.011381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5749-4929-4a0a-9d5a-e900ecd3ef6c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.011648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42e5749-4929-4a0a-9d5a-e900ecd3ef6c None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.012288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e5749-4929-4a0a-9d5a-e900ecd3ef6c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:41.012584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e5749-4929-4a0a-9d5a-e900ecd3ef6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:41.014046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42e5749-4929-4a0a-9d5a-e900ecd3ef6c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:41.019786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 386/769] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 13:58:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:41.021077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ba8f-bbb7-4453-b90b-9f141a33c1e9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.021356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ba8f-bbb7-4453-b90b-9f141a33c1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.021624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa3ba8f-bbb7-4453-b90b-9f141a33c1e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.040502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-caa3ba8f-bbb7-4453-b90b-9f141a33c1e9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:41.041411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-caa3ba8f-bbb7-4453-b90b-9f141a33c1e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:41.078501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c1e2cf-6727-4b83-af70-92e6d055c84d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:41.117520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:41.118016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 387/770] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:41.118866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.119039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3ba8f-bbb7-4453-b90b-9f141a33c1e9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:41.119584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.120125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.128072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:41.129835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:41.141552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:41.142591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:41.143626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:41.143626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:41.143626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:41.143626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:41.143626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:41.143626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:41.143626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:41.143626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:41.150354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 384/771] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:41.153190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:41.183709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:41.190578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:41.259989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74cc71-100a-44c8-820b-d627ea2b35bd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:41.292369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa3ba8f-bbb7-4453-b90b-9f141a33c1e9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:41.295160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:41.296143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.296607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.297058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.297961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:41.298451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:41.300129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:41.313168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 385/772] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:41.319430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:41.327225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:41.327847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.328123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.328391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.329096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] RBAC: Adding query filter params (interface=public, service_id=7179fbdb67d8453bb563256cf613163b, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:41.329519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:41.330840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:41.339832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 388/773] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 13:58:41 2022] GET /identity/v3/endpoints?service_id=7179fbdb67d8453bb563256cf613163b&interface=public®ion_id=RegionOne => generated 194 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:41.350790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:41.421285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:41.435773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e0546f-f23b-4a1a-b51b-c9601ced60bf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:41.471651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:41.472653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.473129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.473582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.474530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=8f18740d784843ada83018867899429a) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:41.475034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:41.476669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:41.483174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 386/774] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:41 2022] GET /identity/v3/role_assignments?user.id=8f18740d784843ada83018867899429a&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:41.515641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:41.556589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce534f-2308-413a-97eb-d6737227a378 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:41.564571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:41.596821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5773fc0-7125-446a-a6f8-7e950006668f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:41.597542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773fc0-7125-446a-a6f8-7e950006668f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.597917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773fc0-7125-446a-a6f8-7e950006668f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.598249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5773fc0-7125-446a-a6f8-7e950006668f None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.598936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5773fc0-7125-446a-a6f8-7e950006668f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:41.600403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5773fc0-7125-446a-a6f8-7e950006668f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:41.609782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 389/775] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 13:58:41 2022] GET /identity/v3/services => generated 772 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:41.727632 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168d3c9-0718-4c08-8afa-71e415358e5c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:41.781036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:41.781733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.782012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.782275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.782871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:41.784299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:41.801305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:41.802422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:41.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 387/776] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:41 2022] GET /identity/v3/services/network => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:41.808871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5773fc0-7125-446a-a6f8-7e950006668f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:41.924251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5773fc0-7125-446a-a6f8-7e950006668f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:41.963799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c692284f-86c8-4a49-8f6c-cad4dc0483f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:41.964418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692284f-86c8-4a49-8f6c-cad4dc0483f2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:41.964690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692284f-86c8-4a49-8f6c-cad4dc0483f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:41.964956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c692284f-86c8-4a49-8f6c-cad4dc0483f2 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:41.965592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692284f-86c8-4a49-8f6c-cad4dc0483f2 None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:41.965918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692284f-86c8-4a49-8f6c-cad4dc0483f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:41.967280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c692284f-86c8-4a49-8f6c-cad4dc0483f2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:41.972646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 390/777] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:58:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:41.978272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:42.013484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:42.122452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5c32d-6313-4d8f-9b01-05b46de47083 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:42.123082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5c32d-6313-4d8f-9b01-05b46de47083 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:42.123689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf5c32d-6313-4d8f-9b01-05b46de47083 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:42.131824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aed732-534d-4950-a286-a76fe8c009b2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:42.132808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 391/778] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:42.137908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6dd4d-2d6d-45da-b62a-63ac37cea878 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:42.138190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6dd4d-2d6d-45da-b62a-63ac37cea878 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:42.138458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e6dd4d-2d6d-45da-b62a-63ac37cea878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:42.176614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-43e6dd4d-2d6d-45da-b62a-63ac37cea878 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:42.177844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-43e6dd4d-2d6d-45da-b62a-63ac37cea878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:42.179562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:42.180186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:42.180463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:42.180733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:42.181373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] RBAC: Adding query filter params (type=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:42.181701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:42.183085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:42.201367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 388/779] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:58:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:42.207459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:42.218320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:42.265780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 392/780] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:58:42.330112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:42.350670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d10014-8bd6-4417-a64b-584a3826c1ac None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:42.390075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e08d0e1-f780-43a0-b532-3503c5eddf70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:42.390735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08d0e1-f780-43a0-b532-3503c5eddf70 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:42.391084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08d0e1-f780-43a0-b532-3503c5eddf70 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:42.391422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e08d0e1-f780-43a0-b532-3503c5eddf70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:42.392221 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08d0e1-f780-43a0-b532-3503c5eddf70 None admin] RBAC: Adding query filter params (interface=public, service_id=e8826b04187e42a4a52245b02f6d71d0, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:42.392613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08d0e1-f780-43a0-b532-3503c5eddf70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:42.394415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e08d0e1-f780-43a0-b532-3503c5eddf70 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:42.408465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 389/781] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 13:58:42 2022] GET /identity/v3/endpoints?service_id=e8826b04187e42a4a52245b02f6d71d0&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:42.410171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6dd4d-2d6d-45da-b62a-63ac37cea878 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:42.521901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e6dd4d-2d6d-45da-b62a-63ac37cea878 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:42.550214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d161c2-7ce7-4624-a4df-6dc2a435a65d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:42.550575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d161c2-7ce7-4624-a4df-6dc2a435a65d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:42.550912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d161c2-7ce7-4624-a4df-6dc2a435a65d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:42.552333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 390/782] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:58:42.557363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e068d-0871-4238-abfe-4af69dbe3f40 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:42.557764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e068d-0871-4238-abfe-4af69dbe3f40 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:42.558100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75e068d-0871-4238-abfe-4af69dbe3f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:42.586826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854aee00-201e-49a8-a728-cfc1ed1351e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:42.587575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854aee00-201e-49a8-a728-cfc1ed1351e5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:42.587903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854aee00-201e-49a8-a728-cfc1ed1351e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:42.588237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854aee00-201e-49a8-a728-cfc1ed1351e5 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:42.588930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854aee00-201e-49a8-a728-cfc1ed1351e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:42.590481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854aee00-201e-49a8-a728-cfc1ed1351e5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:42.591393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e75e068d-0871-4238-abfe-4af69dbe3f40 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:42.593627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e75e068d-0871-4238-abfe-4af69dbe3f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:42.609791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 393/783] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 13:58:42 2022] GET /identity/v3/services => generated 772 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:42.653682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854aee00-201e-49a8-a728-cfc1ed1351e5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:42.681983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:42.724465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 391/784] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:58:42.806146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854aee00-201e-49a8-a728-cfc1ed1351e5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:42.859723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe28461-0ed0-420c-bff5-b8a52cd36e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:42.860383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe28461-0ed0-420c-bff5-b8a52cd36e68 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:42.860668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe28461-0ed0-420c-bff5-b8a52cd36e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:42.860937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe28461-0ed0-420c-bff5-b8a52cd36e68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:42.876938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe28461-0ed0-420c-bff5-b8a52cd36e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bbe28461-0ed0-420c-bff5-b8a52cd36e68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:42.882040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:42.883007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:42.883831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:42.885661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 394/785] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:42.889268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e068d-0871-4238-abfe-4af69dbe3f40 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:42.967753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:42.994926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe28461-0ed0-420c-bff5-b8a52cd36e68 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:42.996241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75e068d-0871-4238-abfe-4af69dbe3f40 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:43.040957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:43.042008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:43.042008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:43.042273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:43.043063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:43.043361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:43.044887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:43.062163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 392/786] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:43.071804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:43.162018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:43.189626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe28461-0ed0-420c-bff5-b8a52cd36e68 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:43.227394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631a8b3-1eb5-44d3-a7a5-de5bffe93367 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:43.255393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983bc01-caec-4697-b1d8-4c3f0812b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:43.258673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983bc01-caec-4697-b1d8-4c3f0812b615 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:43.258949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983bc01-caec-4697-b1d8-4c3f0812b615 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:43.259214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d983bc01-caec-4697-b1d8-4c3f0812b615 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:43.277950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d983bc01-caec-4697-b1d8-4c3f0812b615 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d983bc01-caec-4697-b1d8-4c3f0812b615 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:43.285187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:43.286142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:43.286996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:43.287611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 395/787] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:43.288172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:43.288415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:43.288685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:43.288943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:43.292128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:43.293602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983bc01-caec-4697-b1d8-4c3f0812b615 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:43.310741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:43.314100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:43.339386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:43.340815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:43.341753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:43.341753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:43.341753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:43.341753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 393/788] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:43.346392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:43.485750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d983bc01-caec-4697-b1d8-4c3f0812b615 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:43.540946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1058dc32-bd9b-47de-99b1-ce51ad89bdbb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:43.622668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:43.623312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:43.623586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:43.623850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:43.624509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:43.624806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:43.626235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:43.651187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 394/789] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:43.656555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:43.685447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:43.687322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:43.687994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:43.689616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:43.690621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:43.691834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:43.693822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:43.714176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 396/790] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:43.717115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:43.729255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:43.766071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:43.823040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea65227-a6b4-47e8-8b0c-b6ad09dd362a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:43.888710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf53a31-8efb-481c-9981-212743f78ba3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:43.910938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:43.911996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:43.912471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:43.912931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:43.928814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:43.930328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:43.930984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:43.931259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:43.931527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:43.932716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:43.950948 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:43.952409 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:43.958029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:43.958949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:43.959869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 395/791] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:58:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:43.971301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:43.987768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:43.990007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:43.992039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:43.992039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:43.992039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:43.992039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 397/792] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:44.001757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:44.085414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:44.136594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c0b2d-3c13-4e30-9610-23ecc1505d7d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:44.168485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ede0e-b269-4c3c-8adc-d14a55b70e38 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:44.175736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1303e78c-7777-40e7-ae32-918d5925d65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:44.176354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303e78c-7777-40e7-ae32-918d5925d65f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.176679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303e78c-7777-40e7-ae32-918d5925d65f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.176950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1303e78c-7777-40e7-ae32-918d5925d65f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.177612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303e78c-7777-40e7-ae32-918d5925d65f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:44.177937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303e78c-7777-40e7-ae32-918d5925d65f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:44.179766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1303e78c-7777-40e7-ae32-918d5925d65f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:44.180957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:44.194153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 398/793] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:44.196085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f0b9-ae81-4797-bbe5-8cd8d8ed1e42 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.196366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f0b9-ae81-4797-bbe5-8cd8d8ed1e42 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.196642 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1f0b9-ae81-4797-bbe5-8cd8d8ed1e42 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.202454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 399/794] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:44.202454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1f0b9-ae81-4797-bbe5-8cd8d8ed1e42 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:44.218470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb88c8-6f1f-42ad-9032-699f618e5241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:44.219418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb88c8-6f1f-42ad-9032-699f618e5241 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.219920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb88c8-6f1f-42ad-9032-699f618e5241 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.220409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb88c8-6f1f-42ad-9032-699f618e5241 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.221543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb88c8-6f1f-42ad-9032-699f618e5241 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:44.222084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb88c8-6f1f-42ad-9032-699f618e5241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:44.223780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb88c8-6f1f-42ad-9032-699f618e5241 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:44.243876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 396/795] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:58:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:44.245571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cdb51-41c0-41a3-93a7-68ae3b92139d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.246068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cdb51-41c0-41a3-93a7-68ae3b92139d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.246525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cdb51-41c0-41a3-93a7-68ae3b92139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.265766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-276cdb51-41c0-41a3-93a7-68ae3b92139d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:44.267393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-276cdb51-41c0-41a3-93a7-68ae3b92139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:44.316945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1f0b9-ae81-4797-bbe5-8cd8d8ed1e42 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:44.336731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:44.349935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 397/796] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:44.352252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cdb51-41c0-41a3-93a7-68ae3b92139d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:44.361780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:44.361780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.361780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.361780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.364946 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:44.366436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:44.378611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:44.379528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:44.380452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 400/797] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:44.385664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:44.480144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:44.489125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cdb51-41c0-41a3-93a7-68ae3b92139d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:44.534952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2f57a2-64b2-4dbe-98f9-8ec604861ccd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:44.540348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829c94ec-d987-4a3e-8214-42837d361c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:44.541205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c94ec-d987-4a3e-8214-42837d361c11 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.541750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c94ec-d987-4a3e-8214-42837d361c11 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.542218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829c94ec-d987-4a3e-8214-42837d361c11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.576477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bf99c6-d935-44d6-9069-1135d03aa1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:44.577458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf99c6-d935-44d6-9069-1135d03aa1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.577956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf99c6-d935-44d6-9069-1135d03aa1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.578447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bf99c6-d935-44d6-9069-1135d03aa1b7 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.579378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf99c6-d935-44d6-9069-1135d03aa1b7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:44.579876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf99c6-d935-44d6-9069-1135d03aa1b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:44.582233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bf99c6-d935-44d6-9069-1135d03aa1b7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:44.602030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 401/798] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 13:58:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:44.603590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9063a1-f801-4400-985d-7fdd3701d877 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.604074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9063a1-f801-4400-985d-7fdd3701d877 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.604534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9063a1-f801-4400-985d-7fdd3701d877 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.606209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 402/799] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:44.608250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9063a1-f801-4400-985d-7fdd3701d877 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:44.626197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c94ec-d987-4a3e-8214-42837d361c11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:44.627777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829c94ec-d987-4a3e-8214-42837d361c11 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:44.649086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:44.724358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9063a1-f801-4400-985d-7fdd3701d877 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:44.727042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:44.740079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 398/800] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:44 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:58:44.740079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51005c4-24a4-4060-b31a-54d55ef91bc0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.740079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51005c4-24a4-4060-b31a-54d55ef91bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.740079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51005c4-24a4-4060-b31a-54d55ef91bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.757743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a51005c4-24a4-4060-b31a-54d55ef91bc0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:44.758939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a51005c4-24a4-4060-b31a-54d55ef91bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:44.760392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:44.761033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.761308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.761573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.772156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:44.773588 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:44.788201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:44.789187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:44.790652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:44.790652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:44.790652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:44.790652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 403/801] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:44.790652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:44.841584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 399/802] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:44.842927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51005c4-24a4-4060-b31a-54d55ef91bc0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:44.905680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:44.922723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e092c9-2199-48e6-973f-2641330b36a6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:44.949125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51005c4-24a4-4060-b31a-54d55ef91bc0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:44.965308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6024c6-409e-4a68-b0e9-315ffec75800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:44.966105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6024c6-409e-4a68-b0e9-315ffec75800 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.966459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6024c6-409e-4a68-b0e9-315ffec75800 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.966965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6024c6-409e-4a68-b0e9-315ffec75800 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.976827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:44.980861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6024c6-409e-4a68-b0e9-315ffec75800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-cd6024c6-409e-4a68-b0e9-315ffec75800 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:44.988887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:44.990005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:44.990870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:44.992499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:44.994019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:44.994778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:44.995439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 404/803] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:44.995620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:44.996668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6024c6-409e-4a68-b0e9-315ffec75800 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.009015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.010943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:45.028331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:45.029332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:45.030264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:45.030264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:45.030264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:45.032394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 400/804] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:45.033403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.109926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6024c6-409e-4a68-b0e9-315ffec75800 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.143375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fbce7c-f7b6-4555-9d2d-672c1775df75 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.147554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.148169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.148440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.148700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.149364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:45.149680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.151454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.164836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 405/805] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:45.165570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.177245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:45.180846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03f1b09-9cec-4de5-a5c0-3dcdd639aed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.181459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f1b09-9cec-4de5-a5c0-3dcdd639aed8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.181754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f1b09-9cec-4de5-a5c0-3dcdd639aed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.182014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03f1b09-9cec-4de5-a5c0-3dcdd639aed8 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.182702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f1b09-9cec-4de5-a5c0-3dcdd639aed8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:45.182991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f1b09-9cec-4de5-a5c0-3dcdd639aed8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.184373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03f1b09-9cec-4de5-a5c0-3dcdd639aed8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.197227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 401/806] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:45.197227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514fa32-9409-48b4-b57e-9b061ab17421 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.197227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514fa32-9409-48b4-b57e-9b061ab17421 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.197227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514fa32-9409-48b4-b57e-9b061ab17421 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.197227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 402/807] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:45 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:45.197227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514fa32-9409-48b4-b57e-9b061ab17421 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.238185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:45.266083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa2026d-bde4-4a8a-930e-168528f5be22 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.295495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514fa32-9409-48b4-b57e-9b061ab17421 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.298222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.298834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.299112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.299409 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.300049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:45.300348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.301734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.312539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 406/808] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:45.313324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.330979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.331610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.331884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.332152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.332734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.334121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:45.342698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:45.343749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:45.344564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 403/809] 10.222.0.23 () {56 vars in 1129 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:45.345331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.383931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:45.417026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918eac5-9704-4f98-a6b5-33005a9a87a7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.454436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111f93b-d115-443c-a7e8-848d9bd549b2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.459112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:45.469354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a34e-3850-435c-9a0b-8ae0e9ebfea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.470385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a34e-3850-435c-9a0b-8ae0e9ebfea9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.470802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a34e-3850-435c-9a0b-8ae0e9ebfea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.471201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a34e-3850-435c-9a0b-8ae0e9ebfea9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.479395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a34e-3850-435c-9a0b-8ae0e9ebfea9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.480834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a34e-3850-435c-9a0b-8ae0e9ebfea9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f393a34e-3850-435c-9a0b-8ae0e9ebfea9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:45.492903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:45.494373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 407/810] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:45.495691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bc520-746d-41c4-a8ac-6c394ace6de4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.496228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bc520-746d-41c4-a8ac-6c394ace6de4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.496228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865bc520-746d-41c4-a8ac-6c394ace6de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.499112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.499762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.500029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.500289 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.511580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.513051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.514937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-865bc520-746d-41c4-a8ac-6c394ace6de4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:45.516131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-865bc520-746d-41c4-a8ac-6c394ace6de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:45.527753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:45.528672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:45.534551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:45.534551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:45.534551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:45.534551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 404/811] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:45.534551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.583208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 408/812] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:45.584060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bc520-746d-41c4-a8ac-6c394ace6de4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.644974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:45.647670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017a6472-9863-477a-961c-0d5ea2f84408 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.683921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.684484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.684766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.685035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.690023 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865bc520-746d-41c4-a8ac-6c394ace6de4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.691820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.693283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:45.704947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:45.705918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:45.706806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 405/813] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:45.707305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.726718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:45.732267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.732886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.733141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.733421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.734082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:45.734391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.735730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.745203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 409/814] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:45.745203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.819716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717f6f-52f3-4801-8439-8e1abe73cd45 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.849499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cfa198-9d66-4e20-b134-e3a64ff2145e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.853236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.853876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.854149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.854420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.855101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:45.855402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.856829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.868771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 406/815] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:45.869539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.873479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:45.885360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.885997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:45.886270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:45.886534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:45.887171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:45.887463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:45.888815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:45.899728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 410/816] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:45.900525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:45.942947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:45.965967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29954c9-e99b-401e-9d8a-686ff1faadd6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:45.999326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:45.999934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.000207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.000473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.000677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f748eb5-d9c7-4a72-abed-46ae5d0970dc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.001154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:46.001455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.003531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.014288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 407/817] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:46.015104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.035821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.036436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.036717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.036963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.037584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:46.037924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.039279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.043964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 411/818] 10.222.0.23 () {56 vars in 1124 bytes} [Wed Oct 5 13:58:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:46.044795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.093907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:46.115358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8433064-ae63-4ac6-b7d3-07a43f70252d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.145545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2df0e43-49b6-454b-9c6b-b06ac3e315d6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.148827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.149460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.149768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.150088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] PATH_INFO: `/v3/services/network` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.150764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.152329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.155267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:46.163087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:46.164237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 05 13:58:46.165358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:46.166161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 408/819] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:46 2022] GET /identity/v3/services/network => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:46.166587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.179646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f0b67-e62f-4de9-9267-b9d5c7eee16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.180308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0b67-e62f-4de9-9267-b9d5c7eee16c None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.180582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0b67-e62f-4de9-9267-b9d5c7eee16c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.180856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0b67-e62f-4de9-9267-b9d5c7eee16c None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/01286f6f65b44519b645927f6c70ec5b/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.182938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f0b67-e62f-4de9-9267-b9d5c7eee16c None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=01286f6f65b44519b645927f6c70ec5b, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.184365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f0b67-e62f-4de9-9267-b9d5c7eee16c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.197425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 412/820] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:58:46 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/01286f6f65b44519b645927f6c70ec5b/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:46.198179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f0b67-e62f-4de9-9267-b9d5c7eee16c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.296626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2c6cd0-a7a1-49c7-8f97-a936d7693094 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.320549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f0b67-e62f-4de9-9267-b9d5c7eee16c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.331033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:46.334772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.335430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.335708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.335992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.345577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.347236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:46.360037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:46.361415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:58:46.362474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:46.366791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 409/821] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 13:58:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:46.366923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.367432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.367525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.367706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.367972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.368797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=83f4897f360f4f23aac5749685fdf500) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:46.369108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.370574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.382217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 413/822] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:46 2022] GET /identity/v3/role_assignments?user.id=83f4897f360f4f23aac5749685fdf500&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:46.383087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.442663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:46.499988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3191ea1-4c78-4769-b642-b612e8872b00 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.522763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba633bf-4ea7-40d1-8926-5f07745d6d09 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.540950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484194bb-4533-4a7d-8c19-5e2efa21285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.542033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484194bb-4533-4a7d-8c19-5e2efa21285f None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.542494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484194bb-4533-4a7d-8c19-5e2efa21285f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.542932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484194bb-4533-4a7d-8c19-5e2efa21285f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.543807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484194bb-4533-4a7d-8c19-5e2efa21285f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.545533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484194bb-4533-4a7d-8c19-5e2efa21285f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.559284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.560275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.560716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.561095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.561882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] RBAC: Adding query filter params (name=network) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:46.562145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.563873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.570123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 414/823] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:58:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:46.571014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.573336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 410/824] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 13:58:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:46.581758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484194bb-4533-4a7d-8c19-5e2efa21285f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.660616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:46.733225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484194bb-4533-4a7d-8c19-5e2efa21285f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.748408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543a505c-4be1-48ee-b94c-bf2eb52f2e3c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.754582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:46.786107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.787784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.788271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.788737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.789599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] RBAC: Adding query filter params (type=network) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:46.790120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.791458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.791988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.792095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.792334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.792626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.793318 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:46.794227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.795941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:46.799423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 411/825] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:58:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:46.805188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.823952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 415/826] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 13:58:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:46.830864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:46.917814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4116f42-603a-4db4-9f7e-df578f25f63b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.962056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:46.977790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d0b86e-0a55-439d-bc70-3c24615d4659 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:46.980731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac5314-0998-44bb-8400-252ecba3b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:46.981664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac5314-0998-44bb-8400-252ecba3b88b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:46.982130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac5314-0998-44bb-8400-252ecba3b88b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:46.982581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac5314-0998-44bb-8400-252ecba3b88b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:46.984232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac5314-0998-44bb-8400-252ecba3b88b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:46.991262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caac5314-0998-44bb-8400-252ecba3b88b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:47.025458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 412/827] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 13:58:46 2022] POST /identity/v3/endpoints => generated 337 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:58:47.053699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:47.054725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:47.055212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:47.055693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:47.068662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:47.070751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:47.087028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:47.088126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:47.088985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:47.088985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:47.088985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:47.091441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 416/828] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:47.097735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac5314-0998-44bb-8400-252ecba3b88b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:47.149082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:47.247398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac5314-0998-44bb-8400-252ecba3b88b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:47.284797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:47.285460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:47.285761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:47.286024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:47.286662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:47.286962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:47.288393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:47.300961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 413/829] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:47.306494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:47.410004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c2186-cf81-46db-aaea-9e1c56a1d16d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:47.448066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:47.448703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:47.449029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:47.449297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:47.456647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:47.458129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:47.460759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:47.471548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:47.472543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:47.473437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 417/830] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:47.478803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:47.574614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:47.584571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf302b1-1d68-4593-acea-1f24dd04dcaf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:47.620799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5d9a0-b81c-426d-9c30-79a3a590dd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:47.621420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5d9a0-b81c-426d-9c30-79a3a590dd04 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:47.621745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5d9a0-b81c-426d-9c30-79a3a590dd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:47.622014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5d9a0-b81c-426d-9c30-79a3a590dd04 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:47.622707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5d9a0-b81c-426d-9c30-79a3a590dd04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:47.623006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5d9a0-b81c-426d-9c30-79a3a590dd04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:47.624441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5d9a0-b81c-426d-9c30-79a3a590dd04 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:47.638492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 414/831] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:47.644891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:47.750556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523ef3f-4cda-4d97-8e87-ed143f6da70c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:47.784202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640eb9a-bf41-441c-8b3b-1b81e485192b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:47.784860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640eb9a-bf41-441c-8b3b-1b81e485192b None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:47.785130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640eb9a-bf41-441c-8b3b-1b81e485192b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:47.785393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640eb9a-bf41-441c-8b3b-1b81e485192b None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/8f18740d784843ada83018867899429a/roles/7d39e08123cb40c78586e4d7eb3ed9ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:47.799934 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640eb9a-bf41-441c-8b3b-1b81e485192b None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=8f18740d784843ada83018867899429a, role_id=7d39e08123cb40c78586e4d7eb3ed9ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:47.801358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640eb9a-bf41-441c-8b3b-1b81e485192b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:47.817091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 418/832] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:58:47 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/8f18740d784843ada83018867899429a/roles/7d39e08123cb40c78586e4d7eb3ed9ba => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:58:48.474371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347e664-2033-4779-8f82-c205655bdcfc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:48.474681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347e664-2033-4779-8f82-c205655bdcfc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:48.475022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9347e664-2033-4779-8f82-c205655bdcfc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:48.476442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 415/833] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:48.483157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3ab7-7920-4f3a-bf4d-90127cdd00c1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:48.483717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3ab7-7920-4f3a-bf4d-90127cdd00c1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:48.484183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f3ab7-7920-4f3a-bf4d-90127cdd00c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:48.503900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-661f3ab7-7920-4f3a-bf4d-90127cdd00c1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:48.505490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-661f3ab7-7920-4f3a-bf4d-90127cdd00c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:48.558926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:48.581003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 419/834] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:48.846951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347e664-2033-4779-8f82-c205655bdcfc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:48.955594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9347e664-2033-4779-8f82-c205655bdcfc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:48.990216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94f2c5-3fd6-4280-91d7-c67ef6adb6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:48.990881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94f2c5-3fd6-4280-91d7-c67ef6adb6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:48.991153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94f2c5-3fd6-4280-91d7-c67ef6adb6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:48.991417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf94f2c5-3fd6-4280-91d7-c67ef6adb6d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:48.996394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee3aa-1e43-4328-94e1-78ac3dc6731c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:48.997386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee3aa-1e43-4328-94e1-78ac3dc6731c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:48.997906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09ee3aa-1e43-4328-94e1-78ac3dc6731c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.003995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf94f2c5-3fd6-4280-91d7-c67ef6adb6d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:49.004423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 420/835] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:49.006867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916b80-8412-4227-b518-393c08276094 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.007417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916b80-8412-4227-b518-393c08276094 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.007878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1916b80-8412-4227-b518-393c08276094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-bf94f2c5-3fd6-4280-91d7-c67ef6adb6d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:49.008882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:49.009829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:49.014290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 416/836] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:49.018479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94f2c5-3fd6-4280-91d7-c67ef6adb6d6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.036821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:49.040413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d1916b80-8412-4227-b518-393c08276094 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:49.041955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d1916b80-8412-4227-b518-393c08276094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:49.131062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf94f2c5-3fd6-4280-91d7-c67ef6adb6d6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:49.133671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 421/837] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:49.133835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272723-dd7f-4b25-9fff-6e21314a8572 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.134119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272723-dd7f-4b25-9fff-6e21314a8572 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.134386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9272723-dd7f-4b25-9fff-6e21314a8572 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.139583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 422/838] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:49.140792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c48a0a-a226-4f0d-9e9e-c9ff52faeee7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.141064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c48a0a-a226-4f0d-9e9e-c9ff52faeee7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.141325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c48a0a-a226-4f0d-9e9e-c9ff52faeee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.161198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e2c48a0a-a226-4f0d-9e9e-c9ff52faeee7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:49.162780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e2c48a0a-a226-4f0d-9e9e-c9ff52faeee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:49.180539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:49.181255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.181755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.182090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.183094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:49.183462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:49.184899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:49.195794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 417/839] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:49.202608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.249670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 423/840] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:49.285189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:49.309275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdffd1-a24f-4608-8d10-55a6390f7d6c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:49.342869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:49.343738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.344160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.344574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.355420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:49.357068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:49.371694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:49.372859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:49.373949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:49.373949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:49.373949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:49.380155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 418/841] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:49.381800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c48a0a-a226-4f0d-9e9e-c9ff52faeee7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.404124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.425471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:49.492412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c48a0a-a226-4f0d-9e9e-c9ff52faeee7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:49.525383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6b4e5-7e9d-4c24-9b77-9989d8d97a55 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:49.531427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:49.532306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.532773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.533335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.534288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:49.534783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:49.536611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:49.549978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 424/842] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:49.551261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.554402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:49.562165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:49.562814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.563090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.563361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.563955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:49.565454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:49.574448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:49.575690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:49.576565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 419/843] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:49.577360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.659560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:49.663755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f8e9c8-b20a-4d8a-9f42-9ca932f6ce9c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:49.682771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b02df8-07c7-48f0-b8fb-daa6b483b1eb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:49.702116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:49.702754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.703026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.703289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.703869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:49.705234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:49.714163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:49.715089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:49.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 425/844] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:49.717205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.723711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:49.724361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.724634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.724901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.732628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:49.734111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:49.746068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:49.747186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 420/845] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:49.748303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.800977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:49.828946 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705b6df-62b2-4aec-87ed-2cda488868a9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:49.853073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9b081-3343-4625-8545-daf9a29d8913 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:49.864271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:49.864960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.865234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.865498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.866162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:49.866460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:49.867813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:49.876857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 426/846] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:49.876857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.891556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:49.892184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:49.892466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:49.892737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:49.893415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:49.893744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:49.895132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:49.899525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 421/847] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:49.900270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:49.909871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:49.994915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e5178-6fd6-4aae-819a-d0abe2fd05e9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.011805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d1722e-9f85-4fee-8d1c-29e96b537d0d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.026258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:50.031463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.032085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.032361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.032666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.033378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:50.033739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:50.035174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:50.047784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f234dfc2-5480-4014-9b81-c38ac5a39547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.048397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234dfc2-5480-4014-9b81-c38ac5a39547 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.048671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234dfc2-5480-4014-9b81-c38ac5a39547 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.048940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234dfc2-5480-4014-9b81-c38ac5a39547 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.049627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234dfc2-5480-4014-9b81-c38ac5a39547 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:50.049947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234dfc2-5480-4014-9b81-c38ac5a39547 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:50.051299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f234dfc2-5480-4014-9b81-c38ac5a39547 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:50.052517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 427/848] 10.222.0.23 () {56 vars in 1135 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:50.053322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:50.056055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 422/849] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:50.057280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b4684-ccf5-4b16-88eb-d3b6b053dfd3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.057561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b4684-ccf5-4b16-88eb-d3b6b053dfd3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.057952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b4684-ccf5-4b16-88eb-d3b6b053dfd3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.059319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 423/850] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:50.065193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad7e31-50bd-4cbf-a8ba-9ce3ac52d5a4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.065479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad7e31-50bd-4cbf-a8ba-9ce3ac52d5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.065787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad7e31-50bd-4cbf-a8ba-9ce3ac52d5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.085923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6fad7e31-50bd-4cbf-a8ba-9ce3ac52d5a4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:50.087164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6fad7e31-50bd-4cbf-a8ba-9ce3ac52d5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:50.150458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:50.165688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 424/851] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:58:50.166647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad7e31-50bd-4cbf-a8ba-9ce3ac52d5a4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:50.177005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541852c6-15ff-4eea-b65d-1716b68e77ad None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.222165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.222809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.223083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.223352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.224145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:50.224394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:50.226005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:50.232043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 428/852] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:50.286236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad7e31-50bd-4cbf-a8ba-9ce3ac52d5a4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.298929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:50.328335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e196-0c9a-456a-aa97-de21a6740324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.329179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e196-0c9a-456a-aa97-de21a6740324 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.329574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e196-0c9a-456a-aa97-de21a6740324 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.330014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4e196-0c9a-456a-aa97-de21a6740324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.412750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4e196-0c9a-456a-aa97-de21a6740324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:50.414556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b4e196-0c9a-456a-aa97-de21a6740324 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:50.439722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:50.527984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 425/853] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:58:50.534662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e196-0c9a-456a-aa97-de21a6740324 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:50.547605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:50.563863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c5af-a095-490d-9fc9-678df41b19c5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.599694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08870580-1b41-4ccc-aab2-cd687b0f5a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.600730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870580-1b41-4ccc-aab2-cd687b0f5a19 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.601264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870580-1b41-4ccc-aab2-cd687b0f5a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.601535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08870580-1b41-4ccc-aab2-cd687b0f5a19 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.613250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08870580-1b41-4ccc-aab2-cd687b0f5a19 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-08870580-1b41-4ccc-aab2-cd687b0f5a19 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:50.618721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:50.619866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:50.620995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 429/854] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:50.631069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08870580-1b41-4ccc-aab2-cd687b0f5a19 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:50.647139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:50.657224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b4e196-0c9a-456a-aa97-de21a6740324 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.704400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.706026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.706603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.707168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.721038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:50.722802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:50.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:50.745320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:50.746272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:50.746272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:50.746272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:50.747396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 426/855] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:50.752769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:50.755016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08870580-1b41-4ccc-aab2-cd687b0f5a19 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.789993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.790959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.791408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.791846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.792690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:50.793161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:50.794763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:50.803771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 430/856] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:50.809067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:50.820233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:50.871190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf52839-55fb-420d-ae03-81751719aa82 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.882113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:50.915721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd6f2bd-dab8-43f8-acdf-eb288cbb06b2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:50.918974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.919765 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.920184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.920594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.921388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:50.921862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:50.923920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:50.939957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 427/857] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:50.945400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:50.959359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:50.960044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:50.960381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:50.960714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:50.972664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:50.974454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:50.991722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:50.992700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:50.993941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:50.993941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:50.993941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:50.993941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 431/858] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:50.998473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:51.081896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e7efd1-6917-4464-bfeb-7f84a2eac542 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:51.108729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:51.112095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca0e1a-2d82-4e54-920c-39f60f48ffdb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:51.125817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:51.126575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:51.126855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:51.127123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:51.135628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:51.137888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:51.140045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:51.152998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:51.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:51.154892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:51.154892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:51.154892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:51.154892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:51.154892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:51.154892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:51.154892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:51.154892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:51.156728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:51.157604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:51.159729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 428/859] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:51.160434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:51.160541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:51.161086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:51.162137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:51.162668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:51.164583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:51.179013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 432/860] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:51.184081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:51.289989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be3411e-bd48-4bea-8550-058699b37028 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:51.302790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54c66ca-fc8b-4e0e-a141-2f8672da02ac None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:51.334721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:51.336224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:51.336705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:51.337144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:51.338042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:51.338516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:51.340124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:51.340249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:51.340940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:51.341280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:51.341615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:51.350136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:51.352266 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:51.360825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 429/861] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:51.363072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:51.366741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:51.367663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:51.368984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 433/862] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 13:58:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:51.374578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:51.411962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:51.434260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:51.474725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdea4097-b996-41a1-b066-d4b386923261 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:51.496668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368c636-6f01-48e3-aea2-8a83ec38b81c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:51.512089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:51.512963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:51.513243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:51.513520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:51.514771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=01286f6f65b44519b645927f6c70ec5b) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:51.515131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:51.516846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:51.529024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 430/863] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:51 2022] GET /identity/v3/role_assignments?user.id=01286f6f65b44519b645927f6c70ec5b&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:51.541788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:51.542799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:51.543275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:51.543753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:51.544781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:51.545368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:51.547153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:51.569723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 434/864] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 13:58:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:51.579363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:51.687637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4f774e-4515-4e3a-8142-46cfed7c2d77 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:51.690422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:51.726815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c818bd-ebb7-449c-bc8c-e7e2e59bfd67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:51.727496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c818bd-ebb7-449c-bc8c-e7e2e59bfd67 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:51.727774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c818bd-ebb7-449c-bc8c-e7e2e59bfd67 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:51.728038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c818bd-ebb7-449c-bc8c-e7e2e59bfd67 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:51.798084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c818bd-ebb7-449c-bc8c-e7e2e59bfd67 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:51.799519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c818bd-ebb7-449c-bc8c-e7e2e59bfd67 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:51.903521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 431/865] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:58:51.909736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:52.011434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b966a8a-9d72-4b4c-86fc-9ff77062fa6b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:52.013345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:52.046921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:52.047568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.047843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.048164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.058641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:52.060086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:52.074976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:52.075951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:52.077227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:52.077227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:52.077227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:52.077227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 435/866] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:52.086569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c818bd-ebb7-449c-bc8c-e7e2e59bfd67 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:52.089813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:52.193730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c818bd-ebb7-449c-bc8c-e7e2e59bfd67 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:52.232525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56acc1c9-a6f4-4f99-a05c-bf7dca8cdb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:52.233205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acc1c9-a6f4-4f99-a05c-bf7dca8cdb10 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.233485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acc1c9-a6f4-4f99-a05c-bf7dca8cdb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.233786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56acc1c9-a6f4-4f99-a05c-bf7dca8cdb10 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.234438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acc1c9-a6f4-4f99-a05c-bf7dca8cdb10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:52.234739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acc1c9-a6f4-4f99-a05c-bf7dca8cdb10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:52.236482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56acc1c9-a6f4-4f99-a05c-bf7dca8cdb10 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:52.249280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 432/867] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:52.255004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:52.328165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a972a-19a0-4612-86ce-0ffc0968c117 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.328381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a972a-19a0-4612-86ce-0ffc0968c117 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.328630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a972a-19a0-4612-86ce-0ffc0968c117 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.330267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 433/868] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:58:52.335606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be762b16-02fa-4565-bfa8-08c5fded41c4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.335889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be762b16-02fa-4565-bfa8-08c5fded41c4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.336157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be762b16-02fa-4565-bfa8-08c5fded41c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.354791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-be762b16-02fa-4565-bfa8-08c5fded41c4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:52.355995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-be762b16-02fa-4565-bfa8-08c5fded41c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:52.367829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a571f5-f2b0-420a-95d7-83504252df58 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:52.406735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:52.407629 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.408006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.408370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.416847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:52.418557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:52.433759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:52.434760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:52.435636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:52.435636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:52.435636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:52.435636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:52.435636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:52.435636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:52.435636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:52.435636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:52.438096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:52.440313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 436/869] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:52.441601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:52.457684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 434/870] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:58:52.465257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:52.535053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d9a0-1360-4260-8d0e-615ee40e5542 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.535350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d9a0-1360-4260-8d0e-615ee40e5542 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.535599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d9a0-1360-4260-8d0e-615ee40e5542 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.540923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 435/871] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:58:52.542632 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24323683-fc7d-413c-8f41-b7e0a11e1abd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.542913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24323683-fc7d-413c-8f41-b7e0a11e1abd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.543185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24323683-fc7d-413c-8f41-b7e0a11e1abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.566295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-24323683-fc7d-413c-8f41-b7e0a11e1abd None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:52.566524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a63c81-0511-43a1-b49b-461dba6da7c4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:52.567496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24323683-fc7d-413c-8f41-b7e0a11e1abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:52.603699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:52.604287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.604560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.604825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.605515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:52.605850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:52.607362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:52.620863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 437/872] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:52.627231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:52.649718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 436/873] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:52.708063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:52.733709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038edb60-5bec-4d2a-965b-dbc7d4ad0470 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:52.751384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24323683-fc7d-413c-8f41-b7e0a11e1abd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:52.772488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:52.773880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.774167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.774432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.775192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=8f18740d784843ada83018867899429a) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:52.775500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:52.776922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:52.783094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 438/874] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:52 2022] GET /identity/v3/role_assignments?user.id=8f18740d784843ada83018867899429a&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:52.890261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24323683-fc7d-413c-8f41-b7e0a11e1abd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:52.919835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:52.932000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499fa354-0823-422b-bf1c-394b98b7ce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:52.932631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fa354-0823-422b-bf1c-394b98b7ce71 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:52.932904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fa354-0823-422b-bf1c-394b98b7ce71 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:52.933172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499fa354-0823-422b-bf1c-394b98b7ce71 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:52.933887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fa354-0823-422b-bf1c-394b98b7ce71 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:52.935285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499fa354-0823-422b-bf1c-394b98b7ce71 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:52.940196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:52.954111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 437/875] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:58:52 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:58:53.062962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9b577-256a-4131-bc79-3bee8adc7a1f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:53.108911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db8bee-6ec6-4f8e-8a15-20c500045292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:53.109541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db8bee-6ec6-4f8e-8a15-20c500045292 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:53.109842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db8bee-6ec6-4f8e-8a15-20c500045292 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:53.110105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db8bee-6ec6-4f8e-8a15-20c500045292 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:53.110787 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db8bee-6ec6-4f8e-8a15-20c500045292 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:53.112582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db8bee-6ec6-4f8e-8a15-20c500045292 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:53.116257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:53.132839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 439/876] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:58:52 2022] POST /identity/v3/services => generated 269 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:58:53.885170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0016775-0101-4a98-9bb2-ca765428931b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:53.885905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0016775-0101-4a98-9bb2-ca765428931b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:53.886392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0016775-0101-4a98-9bb2-ca765428931b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:53.888306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 438/877] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:53.894780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844631f6-b733-4f18-9fb0-c9cedb17210b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:53.895773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844631f6-b733-4f18-9fb0-c9cedb17210b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:53.896246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844631f6-b733-4f18-9fb0-c9cedb17210b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:53.915573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-844631f6-b733-4f18-9fb0-c9cedb17210b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:53.917123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-844631f6-b733-4f18-9fb0-c9cedb17210b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:53.988535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 440/878] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:54.255914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0016775-0101-4a98-9bb2-ca765428931b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:54.369166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0016775-0101-4a98-9bb2-ca765428931b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:54.404155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:54.404804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:54.405080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:54.405351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:54.405987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:54.407366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:54.416607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:54.417584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:54.418505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 439/879] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:58:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:54.425882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844631f6-b733-4f18-9fb0-c9cedb17210b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:54.531686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844631f6-b733-4f18-9fb0-c9cedb17210b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:54.556256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:54.574132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaac9c-57b2-470f-b775-b31c25efbafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:54.574746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaac9c-57b2-470f-b775-b31c25efbafa None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:54.575021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaac9c-57b2-470f-b775-b31c25efbafa None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:54.575286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaac9c-57b2-470f-b775-b31c25efbafa None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:54.575912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aaac9c-57b2-470f-b775-b31c25efbafa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:54.576218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aaac9c-57b2-470f-b775-b31c25efbafa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:54.577592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aaac9c-57b2-470f-b775-b31c25efbafa None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:54.582834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 441/880] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:58:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:54.590929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:54.644911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:54.697919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5170d0-5641-48ce-acad-a52916edf228 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:54.731739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a622c46e-caa9-4000-b45a-35dec958aff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:54.732356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c46e-caa9-4000-b45a-35dec958aff1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:54.732631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c46e-caa9-4000-b45a-35dec958aff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:54.732897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a622c46e-caa9-4000-b45a-35dec958aff1 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:54.733530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622c46e-caa9-4000-b45a-35dec958aff1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:54.733855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622c46e-caa9-4000-b45a-35dec958aff1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:54.735224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a622c46e-caa9-4000-b45a-35dec958aff1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:54.740504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 440/881] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:58:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:55.119873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfeda9-532b-439f-85b2-2655c4d0138c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.120432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfeda9-532b-439f-85b2-2655c4d0138c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.120432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfeda9-532b-439f-85b2-2655c4d0138c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.122111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 442/882] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:55.128315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87fffb-d3b5-4ba4-a440-85fcae184dcb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.128669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87fffb-d3b5-4ba4-a440-85fcae184dcb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.128845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf87fffb-d3b5-4ba4-a440-85fcae184dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.148475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bf87fffb-d3b5-4ba4-a440-85fcae184dcb None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:55.149766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf87fffb-d3b5-4ba4-a440-85fcae184dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:55.230769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 441/883] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:58:55.244078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78d73a-9e2a-4a54-bc90-f9a7c07ec14e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.244672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78d73a-9e2a-4a54-bc90-f9a7c07ec14e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.245177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78d73a-9e2a-4a54-bc90-f9a7c07ec14e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.247036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 443/884] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:55.252787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb5363-240d-4fd7-b97c-af1f2506c6c5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.253073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb5363-240d-4fd7-b97c-af1f2506c6c5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.253350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cb5363-240d-4fd7-b97c-af1f2506c6c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.272654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-85cb5363-240d-4fd7-b97c-af1f2506c6c5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:55.273901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-85cb5363-240d-4fd7-b97c-af1f2506c6c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:55.314140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:55.347109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 442/885] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:55.407012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb065c-4550-4133-bb7a-11516da063c2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.407298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb065c-4550-4133-bb7a-11516da063c2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.407567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb065c-4550-4133-bb7a-11516da063c2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.409213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 444/886] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:55.414636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f81aa-ca8f-4eab-97ff-dad428bd99e9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.415139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f81aa-ca8f-4eab-97ff-dad428bd99e9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.415582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2f81aa-ca8f-4eab-97ff-dad428bd99e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.434797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bf2f81aa-ca8f-4eab-97ff-dad428bd99e9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:55.436256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf2f81aa-ca8f-4eab-97ff-dad428bd99e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:55.502900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb065c-4550-4133-bb7a-11516da063c2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:55.521681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 443/887] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:55.609006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb065c-4550-4133-bb7a-11516da063c2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:55.623259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f81aa-ca8f-4eab-97ff-dad428bd99e9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:55.649068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb9a21-61e5-4278-8126-147dc38d6015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:55.649822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb9a21-61e5-4278-8126-147dc38d6015 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.650158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb9a21-61e5-4278-8126-147dc38d6015 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.650435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edb9a21-61e5-4278-8126-147dc38d6015 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.663008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edb9a21-61e5-4278-8126-147dc38d6015 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7edb9a21-61e5-4278-8126-147dc38d6015 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:58:55.668507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:55.669451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:55.670358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 445/888] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 13:58:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:55.675400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb9a21-61e5-4278-8126-147dc38d6015 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:55.727244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:55.749974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2f81aa-ca8f-4eab-97ff-dad428bd99e9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:55.752535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:55.788945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edb9a21-61e5-4278-8126-147dc38d6015 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:55.792123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:55.792769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.793050 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.793315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.793948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:55.795364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:55.804259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:55.805244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:55.806115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 444/889] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:58:55.806885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:55.829675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:55.830329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.830597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.830865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.831604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:55.831833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:55.833360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:55.842263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 446/890] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 13:58:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:55.847634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:55.913432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6701c0d-b2b6-4ab1-961e-bf8665886add None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:55.948402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:55.949037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:55.949306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:55.949570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:55.950238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:55.951618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:55.955849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d33e03-8451-412a-b1c1-ffc33d2bb6bb None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:55.960265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:55.962485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:55.963367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:55.964087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:55.966291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 445/891] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Oct 5 13:58:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:55.967066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:55.998669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:55.999501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.000188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.000492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.000777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.001482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:56.001853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.003697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.014853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 447/892] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:56.014853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.078095 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451f04a-c9b0-42f6-ba6d-66cbd2b5bafe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.112887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.113519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.113950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.114216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.125673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.127346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.128138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b916dc-2e3b-492a-bade-68f51a30423e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:56.142195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:56.143277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:56.144238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:56.144238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:56.144238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:56.144238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 446/893] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:56.144238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.170038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.170582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.170862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.171132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.171762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:56.172070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.173434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.178445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 448/894] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:56.179207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.215883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:56.233327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:56.253286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b542d35-7d85-4339-a0bd-ac71cf41627e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.286766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.287438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.287683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.287891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.288050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad619fc8-38f1-485e-a8bc-a4897e1ffae2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.288511 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:56.288807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.290229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.299905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 447/895] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:56.299905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.334843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.335954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.336235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.336503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.337589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:56.337914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.339982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.360213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 449/896] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:56.360213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.399010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f64bbe8-30ee-4c02-a3b5-306966dc640c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.432954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.434011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.434583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.434852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.435499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:56.435805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.437174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.447411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 448/897] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:56.448458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.457154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:56.463518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:56.464979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219fc9bb-116e-44b1-994b-31de797b5fb2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.505464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.506178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.506640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.506813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.507491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] RBAC: Adding query filter params (interface=public, service_id=d0c4dd84bf5a458fa62ea499664014ac, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:56.507856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.509250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.516574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 450/898] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/endpoints?service_id=d0c4dd84bf5a458fa62ea499664014ac&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:56.517311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.577617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83152f1b-d5b9-4cc5-8dc7-da79d1abb7df None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.613625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.614327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.614546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.614812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.623178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.624625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:58:56.635825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:56.636802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 449/899] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:56.638261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.651908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd42245-bc77-4b9d-8e1c-8f99c0729fab None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.687989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.688523 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.688790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.689057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.689852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] RBAC: Adding query filter params (interface=public, service_id=77b976e746b4418b832ba18a7ed40276, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:56.690160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.691561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.696822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 451/900] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/endpoints?service_id=77b976e746b4418b832ba18a7ed40276&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:56.702048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.707442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:56.726790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:56.740837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d085d9-8e1c-4472-8248-3041c40ebc48 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.776655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24afc6-b110-42be-98dd-007c8a7405d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.777324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24afc6-b110-42be-98dd-007c8a7405d5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.777601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24afc6-b110-42be-98dd-007c8a7405d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.777898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd24afc6-b110-42be-98dd-007c8a7405d5 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.778504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24afc6-b110-42be-98dd-007c8a7405d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.780009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd24afc6-b110-42be-98dd-007c8a7405d5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.793686 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 450/901] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/services => generated 1268 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 13:58:56.793686 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24afc6-b110-42be-98dd-007c8a7405d5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.820393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa1ec93-5a6a-47fb-9594-6f01c0090be4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.859591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.860333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.860672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.861280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.862280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:56.862661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.864221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.892795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 452/902] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:56.893944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:56.928784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd24afc6-b110-42be-98dd-007c8a7405d5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:56.965470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6e05c5-b312-4c2c-8425-11aaf67b28ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:56.966017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e05c5-b312-4c2c-8425-11aaf67b28ba None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.966351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e05c5-b312-4c2c-8425-11aaf67b28ba None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.966754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6e05c5-b312-4c2c-8425-11aaf67b28ba None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.967368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e05c5-b312-4c2c-8425-11aaf67b28ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:56.968830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6e05c5-b312-4c2c-8425-11aaf67b28ba None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:56.970922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:56.976809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 451/903] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/services => generated 1268 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 13:58:56.978382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97a8c7-c381-4d2c-a759-5287c1a11b8c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.978663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97a8c7-c381-4d2c-a759-5287c1a11b8c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.978934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb97a8c7-c381-4d2c-a759-5287c1a11b8c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.984803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 452/904] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:56.987572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ee3b-e53c-4c4f-b6fc-787ae1279bcb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:56.988073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ee3b-e53c-4c4f-b6fc-787ae1279bcb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:56.988492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7ee3b-e53c-4c4f-b6fc-787ae1279bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:56.992205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:57.008859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cdb7ee3b-e53c-4c4f-b6fc-787ae1279bcb None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:57.010130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cdb7ee3b-e53c-4c4f-b6fc-787ae1279bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:57.015776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8825a06a-434e-46bd-8d04-98f35991883e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:57.066491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0171b9e-8b1c-41b7-9db1-7357424bfbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:57.067391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0171b9e-8b1c-41b7-9db1-7357424bfbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:57.067829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0171b9e-8b1c-41b7-9db1-7357424bfbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:57.068247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0171b9e-8b1c-41b7-9db1-7357424bfbc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:57.093773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 453/905] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:57.150641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0171b9e-8b1c-41b7-9db1-7357424bfbc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:57.152312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0171b9e-8b1c-41b7-9db1-7357424bfbc0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:57.235220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:57.256528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 453/906] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:58:56 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:58:57.262676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ee3b-e53c-4c4f-b6fc-787ae1279bcb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:57.355734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0171b9e-8b1c-41b7-9db1-7357424bfbc0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:57.367714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7ee3b-e53c-4c4f-b6fc-787ae1279bcb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:57.400363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:57.412357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:57.413016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:57.413292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:57.413563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:57.432571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:57.434704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:58:57.455202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:57.456193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:58:57.457085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:58:57.457085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:58:57.457085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:57.459507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 454/907] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:58:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:57.465758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:57.477751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0171b9e-8b1c-41b7-9db1-7357424bfbc0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:57.519246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2f09f-2bef-4536-9690-e6f62ea84e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:57.520161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2f09f-2bef-4536-9690-e6f62ea84e7f None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:57.520298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2f09f-2bef-4536-9690-e6f62ea84e7f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:57.520669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b2f09f-2bef-4536-9690-e6f62ea84e7f None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:57.521447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2f09f-2bef-4536-9690-e6f62ea84e7f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:57.522958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b2f09f-2bef-4536-9690-e6f62ea84e7f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:57.545730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 454/908] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:58:57 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:58:57.601281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60336c22-04ee-4285-b40f-9aed9d45657e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:57.647019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:57.647942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:57.648373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:57.648790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:57.649574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:57.650168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:57.651794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:57.666445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 455/909] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:58:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:57.672738 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2f09f-2bef-4536-9690-e6f62ea84e7f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:57.723480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:57.790765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b2f09f-2bef-4536-9690-e6f62ea84e7f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:57.827858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:57.828784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:57.829202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:57.829622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:57.836897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:57.838658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:58:57.850391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:57.851475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:57.852669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:58:57.852669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:58:57.852669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:58:57.852669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:58:57.852669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:58:57.852669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:58:57.852669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:58:57.852669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:58:57.854778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 455/910] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:58:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:57.861291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:57.897434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:57.967747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48191b86-7a0a-435d-8f34-d0bdf4d4c0e6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:58.003291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5620c-6c8b-40d5-b251-e384ba48a108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:58.004183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5620c-6c8b-40d5-b251-e384ba48a108 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:58.004603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5620c-6c8b-40d5-b251-e384ba48a108 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:58.005012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5620c-6c8b-40d5-b251-e384ba48a108 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:58.005871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5620c-6c8b-40d5-b251-e384ba48a108 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:58.006332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5620c-6c8b-40d5-b251-e384ba48a108 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:58.007953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5620c-6c8b-40d5-b251-e384ba48a108 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:58.020471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 456/911] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:58:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:58.026388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:58.127478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:58.131234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587e0c0-59f5-47ee-bf7b-4649277fe47a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:58.172726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa3526-8ce1-4db2-96a0-8a5d73f8f735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:58.174303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3526-8ce1-4db2-96a0-8a5d73f8f735 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:58.174815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3526-8ce1-4db2-96a0-8a5d73f8f735 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:58.175297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa3526-8ce1-4db2-96a0-8a5d73f8f735 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:58.176246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa3526-8ce1-4db2-96a0-8a5d73f8f735 None admin] RBAC: Adding query filter params (role.id=15185fccbb00430dac762cb64ecf6355, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=8f18740d784843ada83018867899429a) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:58.176762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa3526-8ce1-4db2-96a0-8a5d73f8f735 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:58.178479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa3526-8ce1-4db2-96a0-8a5d73f8f735 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:58.185062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 456/912] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:58:58 2022] GET /identity/v3/role_assignments?user.id=8f18740d784843ada83018867899429a&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=15185fccbb00430dac762cb64ecf6355 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:59.057403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8023b-26e1-48fb-a036-c7ea1d5ed948 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.057997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8023b-26e1-48fb-a036-c7ea1d5ed948 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.057997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce8023b-26e1-48fb-a036-c7ea1d5ed948 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.059664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 457/913] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:59.065583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af261-5bd5-4d88-bdba-62ee759aebdf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.065910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af261-5bd5-4d88-bdba-62ee759aebdf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.066158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078af261-5bd5-4d88-bdba-62ee759aebdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.084744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-078af261-5bd5-4d88-bdba-62ee759aebdf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:59.085963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-078af261-5bd5-4d88-bdba-62ee759aebdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:59.162955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 457/914] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:59.218697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f2eb-6bde-42ad-ae21-918e7c1c3dad None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.219020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f2eb-6bde-42ad-ae21-918e7c1c3dad None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.219226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1f2eb-6bde-42ad-ae21-918e7c1c3dad None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.220829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 458/915] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:59.226004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ff18-5def-42ba-bfcb-8a486b8ecc20 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.226459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ff18-5def-42ba-bfcb-8a486b8ecc20 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.226541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf8ff18-5def-42ba-bfcb-8a486b8ecc20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.245200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8bf8ff18-5def-42ba-bfcb-8a486b8ecc20 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:59.246411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8bf8ff18-5def-42ba-bfcb-8a486b8ecc20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:58:59.315446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 458/916] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:58:59.423779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1f2eb-6bde-42ad-ae21-918e7c1c3dad None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:59.525152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:59.533014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1f2eb-6bde-42ad-ae21-918e7c1c3dad None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:59.567980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:59.568549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.568825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.569098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.569721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:59.571227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:59.580237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:59.581218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:59.582076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 459/917] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:59.587229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ff18-5def-42ba-bfcb-8a486b8ecc20 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:59.590910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:59.614420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:59.700553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf8ff18-5def-42ba-bfcb-8a486b8ecc20 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:59.717381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f819a5-6b0c-4e9d-b6cc-3045c788c922 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:59.743412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:59.744682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.745157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.745805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.746770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:59.747266 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:59.749092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:59.759894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:59.759894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.759894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.759894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.760327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:59.760664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 459/918] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:58:59.761675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:59.762732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:58:59.773569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:58:59.774644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:58:59.775503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 460/919] 10.222.0.23 () {56 vars in 1132 bytes} [Wed Oct 5 13:58:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:58:59.785690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:59.808638 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:59.876674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:58:59.890764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202fe336-c59f-4187-b19c-0a6db4e6f6f4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:59.903873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d410b1b7-0b35-4f55-a965-6763b8c77d36 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:58:59.926271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0d821-f9f8-4377-a963-f317dfc1aebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:59.926955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d821-f9f8-4377-a963-f317dfc1aebd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.927232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d821-f9f8-4377-a963-f317dfc1aebd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.927505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0d821-f9f8-4377-a963-f317dfc1aebd None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.928199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0d821-f9f8-4377-a963-f317dfc1aebd None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:59.928512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0d821-f9f8-4377-a963-f317dfc1aebd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:59.929969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0d821-f9f8-4377-a963-f317dfc1aebd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:59.937689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 460/920] 10.222.0.23 () {56 vars in 1114 bytes} [Wed Oct 5 13:58:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:59.937689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b460c5-67a7-4055-bb76-a9435f56e2e5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.937689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b460c5-67a7-4055-bb76-a9435f56e2e5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.937689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b460c5-67a7-4055-bb76-a9435f56e2e5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.938360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 461/921] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:58:59.940690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b460c5-67a7-4055-bb76-a9435f56e2e5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:58:59.945387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c13b7b-a549-4118-a07f-3d6a8ad9862f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:58:59.946043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c13b7b-a549-4118-a07f-3d6a8ad9862f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.946320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c13b7b-a549-4118-a07f-3d6a8ad9862f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.946589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c13b7b-a549-4118-a07f-3d6a8ad9862f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.947230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c13b7b-a549-4118-a07f-3d6a8ad9862f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:58:59.947532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c13b7b-a549-4118-a07f-3d6a8ad9862f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:58:59.948918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c13b7b-a549-4118-a07f-3d6a8ad9862f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:58:59.954267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 461/922] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:58:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:58:59.955555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288cfd3-30b9-4ce9-a15b-c24629e3c686 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:58:59.955832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288cfd3-30b9-4ce9-a15b-c24629e3c686 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:58:59.956111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4288cfd3-30b9-4ce9-a15b-c24629e3c686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:58:59.975298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4288cfd3-30b9-4ce9-a15b-c24629e3c686 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:58:59.976517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4288cfd3-30b9-4ce9-a15b-c24629e3c686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:00.057660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 462/923] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:58:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:00.057660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288cfd3-30b9-4ce9-a15b-c24629e3c686 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:00.061872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b460c5-67a7-4055-bb76-a9435f56e2e5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:00.075265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:00.102346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10250133-263d-4cc9-b4dd-23d58dbf70ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:00.103050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10250133-263d-4cc9-b4dd-23d58dbf70ba None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.103439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10250133-263d-4cc9-b4dd-23d58dbf70ba None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.103780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10250133-263d-4cc9-b4dd-23d58dbf70ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:00.104567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10250133-263d-4cc9-b4dd-23d58dbf70ba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:00.106108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10250133-263d-4cc9-b4dd-23d58dbf70ba None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:00.138195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 462/924] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 13:58:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:59:00.175582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4288cfd3-30b9-4ce9-a15b-c24629e3c686 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:00.222462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd247aeb-de1b-41db-81d2-dd8ef4f5feb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:00.223134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd247aeb-de1b-41db-81d2-dd8ef4f5feb0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.223408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd247aeb-de1b-41db-81d2-dd8ef4f5feb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.223683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd247aeb-de1b-41db-81d2-dd8ef4f5feb0 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:00.224322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd247aeb-de1b-41db-81d2-dd8ef4f5feb0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:00.224614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd247aeb-de1b-41db-81d2-dd8ef4f5feb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:00.226059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd247aeb-de1b-41db-81d2-dd8ef4f5feb0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:00.232356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 463/925] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:59:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:00.239131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10250133-263d-4cc9-b4dd-23d58dbf70ba None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:00.254239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:00.366770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10250133-263d-4cc9-b4dd-23d58dbf70ba None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:00.375009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03480-c207-4901-b907-4cc5dea352ef None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.375288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03480-c207-4901-b907-4cc5dea352ef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.375551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d03480-c207-4901-b907-4cc5dea352ef None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:00.376916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 464/926] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:00.382327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c2b45-dab1-4631-8d72-6249357fa730 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.382599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c2b45-dab1-4631-8d72-6249357fa730 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.382862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01c2b45-dab1-4631-8d72-6249357fa730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:00.404048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f5be9-fecc-4b83-ab68-d04a027e18a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:00.404710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f5be9-fecc-4b83-ab68-d04a027e18a7 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.404921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c01c2b45-dab1-4631-8d72-6249357fa730 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:00.405049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f5be9-fecc-4b83-ab68-d04a027e18a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.405238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f5be9-fecc-4b83-ab68-d04a027e18a7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:00.407366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f5be9-fecc-4b83-ab68-d04a027e18a7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:00.407474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c01c2b45-dab1-4631-8d72-6249357fa730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:00.407550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f5be9-fecc-4b83-ab68-d04a027e18a7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:00.431078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 463/927] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 13:59:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:00.431938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f5be9-fecc-4b83-ab68-d04a027e18a7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:00.502846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:00.516238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 465/928] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:00.547883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f5be9-fecc-4b83-ab68-d04a027e18a7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:00.589655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:00.590464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.590744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.591012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:00.591620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:00.593047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:00.602819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:00.603898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:00.604723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 464/929] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:59:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:00.610410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c2b45-dab1-4631-8d72-6249357fa730 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:00.626385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:00.728365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01c2b45-dab1-4631-8d72-6249357fa730 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:00.762425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:00.763055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.763327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.763594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:00.764233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:00.764534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:00.766010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:00.771239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 466/930] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:59:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:00.776935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:00.802736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:00.891363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518d2c6-f918-4659-a58d-cdd3daf41a76 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:00.892957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:00.922597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:00.931855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:00.932647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.933017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.933478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:00.934357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:00.934725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:00.936156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:00.937849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95bbc14-c5c4-4cee-91da-0aa55fb52cab None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:00.942066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 465/931] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:59:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:00.950880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:00.987837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57d41-4969-4836-bc6a-518d93f20a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:00.989445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57d41-4969-4836-bc6a-518d93f20a50 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:00.989980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57d41-4969-4836-bc6a-518d93f20a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:00.990773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57d41-4969-4836-bc6a-518d93f20a50 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:01.005914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f57d41-4969-4836-bc6a-518d93f20a50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b8f57d41-4969-4836-bc6a-518d93f20a50 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:01.014766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:01.015740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:01.016579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:01.019914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 467/932] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 13:59:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:59:01.025929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57d41-4969-4836-bc6a-518d93f20a50 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:01.071349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec2d536-cefc-4a74-8ca5-eb8f7668d142 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:01.113214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:01.114125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:01.114546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:01.114988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:01.115855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] RBAC: Adding query filter params (interface=public, service_id=4d2e58abc53a41f68761e5c84fcf6a7f, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:01.116323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:01.118031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:01.125122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 466/933] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 13:59:00 2022] GET /identity/v3/endpoints?service_id=4d2e58abc53a41f68761e5c84fcf6a7f&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:01.132416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:01.150381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57d41-4969-4836-bc6a-518d93f20a50 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:01.178615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:01.187783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:01.188400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:01.188679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:01.188954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:01.189767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:01.190080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:01.191558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:01.199929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 468/934] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 13:59:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:01.201372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:01.212317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:01.254635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bd0111-40ec-43fe-a0ac-6877c310cab2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:01.300463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c592305-a2c7-48c6-b30c-2c0a05539aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:01.301271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c592305-a2c7-48c6-b30c-2c0a05539aff None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:01.305688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c592305-a2c7-48c6-b30c-2c0a05539aff None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:01.305688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c592305-a2c7-48c6-b30c-2c0a05539aff None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:01.305688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c592305-a2c7-48c6-b30c-2c0a05539aff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:01.305688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c592305-a2c7-48c6-b30c-2c0a05539aff None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:01.316329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 467/935] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 13:59:01 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 13:59:01.347902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac3eba-80f1-454d-bdab-c65e2ed4b2c7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:01.386867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:01.387822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:01.388248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:01.388668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:01.401390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:01.403251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:01.419757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:01.420970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:01.422098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:01.422098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:01.422098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:01.424101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 469/936] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:01.430015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c592305-a2c7-48c6-b30c-2c0a05539aff None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:01.542442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c592305-a2c7-48c6-b30c-2c0a05539aff None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:01.564950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:01.580421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:01.581049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:01.581323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:01.581591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:01.582310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:01.582580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:01.583977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:01.596067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 468/937] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:01.601714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:01.644445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:01.700706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c005d-4ee8-43c9-b8f8-443eefcd47b0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:01.739344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:01.739990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:01.740259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:01.740531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:01.749227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:01.750702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:59:01.761832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:01.762774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:01.764363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 470/938] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 13:59:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:01.768799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:01.870622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6358d-e160-4e03-b33d-33c8636ad90b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:01.904438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:01.905055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:01.905328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:01.905601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:01.906360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:01.906661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:01.908106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:01.930006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 469/939] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 13:59:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:01.931762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:02.039770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535c9e75-20b9-46b4-a755-71f28bd6b326 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:02.041704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:02.076443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:02.077089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.077368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.077657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.088645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:02.090140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:02.105088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:02.106109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:02.106981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:02.106981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:02.106981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:02.106981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 471/940] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:02.113861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:02.157027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:02.224761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bebe57-3655-4b84-9006-a91fd8f45c0c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:02.261251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:02.262630 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.263058 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.263485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.264275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:02.264724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:02.266335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:02.278949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 470/941] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:02.286778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:02.403973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42c1eb-52d9-4c1c-9615-230155d8110e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:02.438143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb856fc-8fab-4e9a-b519-ac4ce1543305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:02.438798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb856fc-8fab-4e9a-b519-ac4ce1543305 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.439078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb856fc-8fab-4e9a-b519-ac4ce1543305 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.439396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb856fc-8fab-4e9a-b519-ac4ce1543305 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.446454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb856fc-8fab-4e9a-b519-ac4ce1543305 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:02.447906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb856fc-8fab-4e9a-b519-ac4ce1543305 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8cb856fc-8fab-4e9a-b519-ac4ce1543305 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:59:02.459312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:02.460296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:02.461502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 472/942] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:02.466391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:02.495705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1237-34c6-452b-8372-2d7a4f181e85 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.495988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1237-34c6-452b-8372-2d7a4f181e85 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.496263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1237-34c6-452b-8372-2d7a4f181e85 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.497875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 473/943] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:02.511858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded386-7f8a-4f33-954f-e8fec95e2d7d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.512137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded386-7f8a-4f33-954f-e8fec95e2d7d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.512406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdded386-7f8a-4f33-954f-e8fec95e2d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.535578 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fdded386-7f8a-4f33-954f-e8fec95e2d7d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:02.537045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fdded386-7f8a-4f33-954f-e8fec95e2d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:02.548807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:02.579689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:02.586696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74916ade-04a4-4898-8f9f-17c260027807 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:02.622685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 474/944] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:59:02.638323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0944fb-ad56-4cfc-ae5f-84b29ac274b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:02.639348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0944fb-ad56-4cfc-ae5f-84b29ac274b7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.639837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0944fb-ad56-4cfc-ae5f-84b29ac274b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.640309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0944fb-ad56-4cfc-ae5f-84b29ac274b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.641201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0944fb-ad56-4cfc-ae5f-84b29ac274b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:02.641789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0944fb-ad56-4cfc-ae5f-84b29ac274b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:02.643476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0944fb-ad56-4cfc-ae5f-84b29ac274b7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:02.657285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 471/945] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:59:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:02.663697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdded386-7f8a-4f33-954f-e8fec95e2d7d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:02.725791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78d00-d47c-47a1-9cc8-7a2c4b42fb11 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.726126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78d00-d47c-47a1-9cc8-7a2c4b42fb11 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.726320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a78d00-d47c-47a1-9cc8-7a2c4b42fb11 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.727885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 472/946] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:02.732690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269770-00bc-4d6c-8ae5-211362d0082b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.732961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269770-00bc-4d6c-8ae5-211362d0082b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.733225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87269770-00bc-4d6c-8ae5-211362d0082b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.752104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-87269770-00bc-4d6c-8ae5-211362d0082b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:02.753284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87269770-00bc-4d6c-8ae5-211362d0082b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:02.774481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdded386-7f8a-4f33-954f-e8fec95e2d7d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:02.819928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14505b6d-4e55-4d4d-a6e3-9e613ff8f425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:02.821169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14505b6d-4e55-4d4d-a6e3-9e613ff8f425 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:02.821707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14505b6d-4e55-4d4d-a6e3-9e613ff8f425 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:02.822280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14505b6d-4e55-4d4d-a6e3-9e613ff8f425 None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/8f18740d784843ada83018867899429a/roles/15185fccbb00430dac762cb64ecf6355` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:02.824863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14505b6d-4e55-4d4d-a6e3-9e613ff8f425 None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=8f18740d784843ada83018867899429a, role_id=15185fccbb00430dac762cb64ecf6355)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:02.826675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14505b6d-4e55-4d4d-a6e3-9e613ff8f425 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:02.829281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 473/947] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:59:02.842206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 475/948] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:59:02 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/8f18740d784843ada83018867899429a/roles/15185fccbb00430dac762cb64ecf6355 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:59:02.909028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269770-00bc-4d6c-8ae5-211362d0082b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:03.010276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:03.024636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87269770-00bc-4d6c-8ae5-211362d0082b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:03.061294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:03.061947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:03.062222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:03.062489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:03.063140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:03.064653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:03.067317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING keystone.server.flask.common [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 05 13:59:03.085832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.sql.core [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 05 13:59:03.085832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 05 13:59:03.085832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [parameters: {'id': 'bc91ecc2b1404fd9906614095304e69e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 05 13:59:03.085832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81571) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc91ecc2b1404fd9906614095304e69e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc91ecc2b1404fd9906614095304e69e. Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 13:59:03.093387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 05 13:59:03.094205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application yield resource Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.rollback() Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.force_reraise() Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise self.value Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application session.commit() Oct 05 13:59:03.095161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._prepare_impl() Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.flush() Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._flush(objects) Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application compat.raise_( Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application flush_context.execute() Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 05 13:59:03.096101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rec.execute(self) Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 05 13:59:03.096971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 13:59:03.099031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [parameters: {'id': 'bc91ecc2b1404fd9906614095304e69e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:03.100541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:03.101324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bc91ecc2b1404fd9906614095304e69e. Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:03.102301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 474/949] 10.222.0.23 () {60 vars in 1140 bytes} [Wed Oct 5 13:59:02 2022] POST /identity/v3/projects => generated 262 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:03.102839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14505b6d-4e55-4d4d-a6e3-9e613ff8f425 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:03.102911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:03.105072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:03.211067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14505b6d-4e55-4d4d-a6e3-9e613ff8f425 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:03.223633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f4cc3e-e4ee-4a4d-855c-aa528a248e0b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:03.255359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:03.256013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:03.256292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:03.256559 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:03.257355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:03.258507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:03.258756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:03.259349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:03.259718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:03.260081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:03.267265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:03.268712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:03.269880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:03.270734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 476/950] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:03.275165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:03.275905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:59:03.287545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:03.288656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:03.289756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:59:03.289756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:59:03.289756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:59:03.289756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:59:03.289756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:59:03.289756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:59:03.289756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:03.289756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:03.295397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 475/951] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:03.296316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:03.306402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:03.354727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:03.393535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de58bad-3ac2-4ab0-b141-eb720c39ae9a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:03.431477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:03.432190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:03.432526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:03.432857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:03.433623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:03.433999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:03.435461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:03.440659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 477/952] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:59:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:03.442999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cfc1df-63da-4732-a930-7657a28cedb9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:03.446301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:03.487647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16065c11-4574-475e-919f-13e85eb08e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:03.488558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065c11-4574-475e-919f-13e85eb08e40 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:03.489037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065c11-4574-475e-919f-13e85eb08e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:03.489481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16065c11-4574-475e-919f-13e85eb08e40 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:03.490410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16065c11-4574-475e-919f-13e85eb08e40 None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:03.490882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16065c11-4574-475e-919f-13e85eb08e40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:03.492510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16065c11-4574-475e-919f-13e85eb08e40 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:03.515241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 476/953] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:59:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:03.587277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072c14c8-d376-44b1-8aa6-81f24c5ef74a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:03.620367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:03.627041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a89e0d-edfe-4bc7-be09-4b1d097eb3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:03.628285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a89e0d-edfe-4bc7-be09-4b1d097eb3ff None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:03.628565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a89e0d-edfe-4bc7-be09-4b1d097eb3ff None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:03.628830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a89e0d-edfe-4bc7-be09-4b1d097eb3ff None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:03.629472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a89e0d-edfe-4bc7-be09-4b1d097eb3ff None admin] RBAC: Adding query filter params (type=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:03.629816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a89e0d-edfe-4bc7-be09-4b1d097eb3ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:03.631216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a89e0d-edfe-4bc7-be09-4b1d097eb3ff None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:03.636807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 478/954] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:59:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:03.653372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe969db-be5e-4128-8f85-92ed3043e4de None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:03.653682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe969db-be5e-4128-8f85-92ed3043e4de None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:03.653946 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe969db-be5e-4128-8f85-92ed3043e4de None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:03.655743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 477/955] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:03.662659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cdae-0671-4842-8378-15956dd94ac1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:03.663020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cdae-0671-4842-8378-15956dd94ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:03.663217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e2cdae-0671-4842-8378-15956dd94ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:03.683544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b7e2cdae-0671-4842-8378-15956dd94ac1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:03.684793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b7e2cdae-0671-4842-8378-15956dd94ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:03.767309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 479/956] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:03 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:04.031023 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe969db-be5e-4128-8f85-92ed3043e4de None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:04.097060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:04.141794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe969db-be5e-4128-8f85-92ed3043e4de None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:04.179690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:04.180735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:04.181226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:04.181725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:04.182507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:04.184122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:04.193737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:04.194851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:04.195780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:04.198593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 478/957] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:59:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:04.206960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2cdae-0671-4842-8378-15956dd94ac1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:04.306990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e2cdae-0671-4842-8378-15956dd94ac1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:04.333316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:04.342813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:04.343758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:04.344243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:04.344907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:04.345844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:04.346348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:04.347985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:04.353505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 480/958] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:59:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:04.362960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:04.473851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f19ce8-4fad-4da9-8609-8f938c803ea5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:04.501918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:04.511284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3d359c-809d-45c1-8329-f86e65e47137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:04.512694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d359c-809d-45c1-8329-f86e65e47137 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:04.513195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d359c-809d-45c1-8329-f86e65e47137 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:04.513663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3d359c-809d-45c1-8329-f86e65e47137 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:04.514540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d359c-809d-45c1-8329-f86e65e47137 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:04.515079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d359c-809d-45c1-8329-f86e65e47137 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:04.516670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3d359c-809d-45c1-8329-f86e65e47137 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:04.527358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 479/959] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:59:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:04.529115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:04.639504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333e5b37-816a-47b9-b275-bc53a0d5182d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:04.675044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef8f877-d8cc-431b-86bc-0c54ee77a9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:04.676008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8f877-d8cc-431b-86bc-0c54ee77a9d8 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:04.676483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8f877-d8cc-431b-86bc-0c54ee77a9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:04.676928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef8f877-d8cc-431b-86bc-0c54ee77a9d8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:04.677814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8f877-d8cc-431b-86bc-0c54ee77a9d8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:04.679378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef8f877-d8cc-431b-86bc-0c54ee77a9d8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:04.709844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 481/960] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 13:59:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:05.120966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882fad9-26f6-4251-827c-841e72c2d0e6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.121385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882fad9-26f6-4251-827c-841e72c2d0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.121559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882fad9-26f6-4251-827c-841e72c2d0e6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.123035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 480/961] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:05.128367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ea6b9-a6af-4d3d-bb4c-462d8a3f091c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.128654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ea6b9-a6af-4d3d-bb4c-462d8a3f091c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.128926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ea6b9-a6af-4d3d-bb4c-462d8a3f091c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.148110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a64ea6b9-a6af-4d3d-bb4c-462d8a3f091c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:05.149301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a64ea6b9-a6af-4d3d-bb4c-462d8a3f091c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:05.193622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:05.226090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 482/962] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:05.489357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882fad9-26f6-4251-827c-841e72c2d0e6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:05.597092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882fad9-26f6-4251-827c-841e72c2d0e6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:05.633706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da2ba1-9279-461a-9ab6-0434c90c3012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:05.634357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da2ba1-9279-461a-9ab6-0434c90c3012 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.634633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da2ba1-9279-461a-9ab6-0434c90c3012 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.634902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86da2ba1-9279-461a-9ab6-0434c90c3012 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.646367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86da2ba1-9279-461a-9ab6-0434c90c3012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-86da2ba1-9279-461a-9ab6-0434c90c3012 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:05.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:05.652164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:05.652997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 481/963] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 13:59:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:05.658425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ea6b9-a6af-4d3d-bb4c-462d8a3f091c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:05.759740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ea6b9-a6af-4d3d-bb4c-462d8a3f091c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:05.792786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5097fb-6c4a-4c98-8c6d-300ca4a65a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:05.793408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5097fb-6c4a-4c98-8c6d-300ca4a65a48 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.793699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5097fb-6c4a-4c98-8c6d-300ca4a65a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.793967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5097fb-6c4a-4c98-8c6d-300ca4a65a48 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.794632 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5097fb-6c4a-4c98-8c6d-300ca4a65a48 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:05.794928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5097fb-6c4a-4c98-8c6d-300ca4a65a48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:05.796313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5097fb-6c4a-4c98-8c6d-300ca4a65a48 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:05.805953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 483/964] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 13:59:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:05.811911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da2ba1-9279-461a-9ab6-0434c90c3012 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:05.838602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ac30-dec3-44ed-a2b1-b5760efab1ec None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.838919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ac30-dec3-44ed-a2b1-b5760efab1ec None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.839131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227ac30-dec3-44ed-a2b1-b5760efab1ec None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.840703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 484/965] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:05.845778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b1251-7253-4625-b7cc-eb25de46638a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.846059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b1251-7253-4625-b7cc-eb25de46638a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.846327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b1251-7253-4625-b7cc-eb25de46638a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.871727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3f9b1251-7253-4625-b7cc-eb25de46638a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:05.873542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3f9b1251-7253-4625-b7cc-eb25de46638a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:05.884195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:05.904379 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:05.939603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86da2ba1-9279-461a-9ab6-0434c90c3012 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:05.945674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 485/966] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:59:05.951683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba360-fcac-4366-82e3-767c9a93c484 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.952210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba360-fcac-4366-82e3-767c9a93c484 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.952936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ba360-fcac-4366-82e3-767c9a93c484 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.955502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 486/967] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:05.963560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049df8d-9253-4b02-a447-af5128b6e965 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.964089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049df8d-9253-4b02-a447-af5128b6e965 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.964759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049df8d-9253-4b02-a447-af5128b6e965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.977549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:05.978259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:05.978538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:05.978872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:05.988170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0049df8d-9253-4b02-a447-af5128b6e965 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:05.989731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0049df8d-9253-4b02-a447-af5128b6e965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:05.990896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:05.992372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:06.007099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:06.008069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:06.013620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:06.013620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:06.013620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:06.013620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 482/968] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:06.014700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.081658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 487/969] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:06.128449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc21653-5557-436a-93b4-6112ec2b8ce0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:06.130796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:06.162734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:06.163348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.163627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.163897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.164537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:06.164839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:06.166280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.182795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 483/970] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:06.188304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049df8d-9253-4b02-a447-af5128b6e965 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.214115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.298110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:06.307258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049df8d-9253-4b02-a447-af5128b6e965 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:06.331055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221843e-e5c6-426c-a08a-c66f1739ab26 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:06.348935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:06.349580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.349932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.350359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.359726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:06.362024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.370864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:06.371478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.371755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.372022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.372671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:59:06.373529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:06.374430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 05 13:59:06.380505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:06.381069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.381141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 488/971] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 13:59:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:06.381141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.394031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.sql.core [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 05 13:59:06.394031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 05 13:59:06.394031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [parameters: {'user_id': '2d6d000ffde3401f91e517d6d1690a65', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 05 13:59:06.394031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81571) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Oct 05 13:59:06.401471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 13:59:06.402578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application yield resource Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application next(self.gen) Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.rollback() Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.force_reraise() Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise self.value Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application session.commit() Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 05 13:59:06.403778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._prepare_impl() Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.session.flush() Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._flush(objects) Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application compat.raise_( Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application flush_context.execute() Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rec.execute(self) Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 05 13:59:06.404865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 05 13:59:06.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result = self._query(query) Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application conn.query(q) Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application result.read() Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 05 13:59:06.407035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [parameters: {'user_id': '2d6d000ffde3401f91e517d6d1690a65', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:06.408149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 05 13:59:06.409115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 05 13:59:06.410343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 05 13:59:06.410343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 05 13:59:06.410343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 05 13:59:06.410343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 05 13:59:06.410343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 05 13:59:06.410343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:06.410754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 484/972] 10.222.0.23 () {60 vars in 1131 bytes} [Wed Oct 5 13:59:06 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:59:06.411827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.440604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:06.497164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162bc834-1b05-4543-830c-e685bf25274d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:06.528902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfd351-3945-497c-8ed1-092a4d44ce71 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:06.533309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866fdb0-d848-46d8-a7ad-3fe9cad15ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:06.533957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866fdb0-d848-46d8-a7ad-3fe9cad15ddb None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.534236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866fdb0-d848-46d8-a7ad-3fe9cad15ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.534510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866fdb0-d848-46d8-a7ad-3fe9cad15ddb None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.535164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866fdb0-d848-46d8-a7ad-3fe9cad15ddb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:06.536527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c866fdb0-d848-46d8-a7ad-3fe9cad15ddb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.557868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 489/973] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:59:06 2022] POST /identity/v3/services => generated 242 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:06.557868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866fdb0-d848-46d8-a7ad-3fe9cad15ddb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.578354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:06.578715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:06.579032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.579622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.579622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.580619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:06.580989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:06.584668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.606306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 485/974] 10.222.0.23 () {56 vars in 1149 bytes} [Wed Oct 5 13:59:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:06.612331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.709847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866fdb0-d848-46d8-a7ad-3fe9cad15ddb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:06.710933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:06.732092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bad9c-634c-4ada-acfa-4244985aa34c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:06.759246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:06.760593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.761034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.761698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.774611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:06.775749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bfbca7-d26e-488e-b5a8-4b454dd722ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:06.776128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.776379 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfbca7-d26e-488e-b5a8-4b454dd722ae None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.776659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfbca7-d26e-488e-b5a8-4b454dd722ae None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.776924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bfbca7-d26e-488e-b5a8-4b454dd722ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:59:06.791185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:06.792194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:06.793122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 490/975] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:59:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:06.798700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.857495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfbca7-d26e-488e-b5a8-4b454dd722ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:06.859244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bfbca7-d26e-488e-b5a8-4b454dd722ae None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.891288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:06.916794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e066de17-6038-4d35-862e-98e44be321ef None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:06.944228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:06.951518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aedd1e-210b-4a63-bea9-b4f5818ab0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:06.952438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aedd1e-210b-4a63-bea9-b4f5818ab0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.952934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aedd1e-210b-4a63-bea9-b4f5818ab0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.953414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aedd1e-210b-4a63-bea9-b4f5818ab0f1 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.954341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aedd1e-210b-4a63-bea9-b4f5818ab0f1 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:06.954862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aedd1e-210b-4a63-bea9-b4f5818ab0f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:06.956509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aedd1e-210b-4a63-bea9-b4f5818ab0f1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:06.966160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 486/976] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:06 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 05 13:59:06.971729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504966dc-5795-465f-8269-db8603aaebbf None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:06.972006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504966dc-5795-465f-8269-db8603aaebbf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:06.972270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504966dc-5795-465f-8269-db8603aaebbf None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:06.973558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 487/977] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:06.974265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504966dc-5795-465f-8269-db8603aaebbf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:06.999880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 491/978] 10.222.0.23 () {56 vars in 1101 bytes} [Wed Oct 5 13:59:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:07.001693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6c265-7a28-4dd7-b473-714506788277 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.002185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6c265-7a28-4dd7-b473-714506788277 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.002660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad6c265-7a28-4dd7-b473-714506788277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.026782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bad6c265-7a28-4dd7-b473-714506788277 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:07.028493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bad6c265-7a28-4dd7-b473-714506788277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:07.111735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504966dc-5795-465f-8269-db8603aaebbf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:07.117472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 492/979] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:07.150960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:07.151663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.151941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.152467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.165076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:07.166646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:07.183378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:07.184424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:07.185823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:07.185823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:07.185823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:07.185823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 488/980] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:07.190345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6c265-7a28-4dd7-b473-714506788277 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:07.254101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:07.295122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad6c265-7a28-4dd7-b473-714506788277 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:07.329607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:07.330309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.330584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.330848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.331490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:07.331786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:07.333148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:07.345966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 493/981] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:07.350912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:07.383093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:07.384405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:07.455383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e4409-3468-4cda-854d-7c522c75255a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:07.489386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:07.490096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.490539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.490811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.498363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:07.499738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe76925-ec84-4fa3-8921-e5ab28d51ba1 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:07.499872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:59:07.518268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:07.519190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:07.520720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 489/982] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:07.525046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:07.532699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:07.546207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:07.546839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.547111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.547370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.547973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:07.549374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:07.558996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:07.560062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:07.560898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 494/983] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:59:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:07.566024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:07.625219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:07.635022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e362ca9-f36c-4db9-a91a-f2d82f00ba63 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:07.671163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:07.671786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.672065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.672316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.673006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:07.673310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:07.674799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:07.676126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edafb9e-94f1-4283-808c-39c879c4e35e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:07.688218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 490/984] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:59:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:07.694159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:07.712602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:07.713203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.713482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.713820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.714447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:07.714747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:07.716094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:07.721413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 495/985] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:59:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:07.726714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:07.777938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:07.803050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86eca39-2b3e-4001-b22d-68026a8d67ca None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:07.831591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ccbcb1-6d20-4279-b903-6d370a30d6b7 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:07.841482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:07.842132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.842805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.843078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.844096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] RBAC: Adding query filter params (role.id=15185fccbb00430dac762cb64ecf6355, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=8f18740d784843ada83018867899429a) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:07.844399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:07.846120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:07.852518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 491/986] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:07 2022] GET /identity/v3/role_assignments?user.id=8f18740d784843ada83018867899429a&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=15185fccbb00430dac762cb64ecf6355 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:07.879195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:07.880198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:07.880657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:07.881104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:07.881991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:07.882501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:07.884119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:07.895059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 496/987] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:59:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:07.896460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:07.918082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:08.008917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0953c13-9e1c-4112-9c78-900aac3ca847 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:08.047808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07501c-f56e-4426-9ffe-8490a99fc655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:08.048468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07501c-f56e-4426-9ffe-8490a99fc655 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:08.048778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07501c-f56e-4426-9ffe-8490a99fc655 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:08.049047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07501c-f56e-4426-9ffe-8490a99fc655 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:08.049847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07501c-f56e-4426-9ffe-8490a99fc655 None admin] RBAC: Adding query filter params (interface=admin, service_id=4d2e58abc53a41f68761e5c84fcf6a7f, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:08.050156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07501c-f56e-4426-9ffe-8490a99fc655 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:08.051571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07501c-f56e-4426-9ffe-8490a99fc655 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:08.057319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 492/988] 10.222.0.23 () {56 vars in 1249 bytes} [Wed Oct 5 13:59:07 2022] GET /identity/v3/endpoints?service_id=4d2e58abc53a41f68761e5c84fcf6a7f&interface=admin®ion_id=RegionOne => generated 193 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:08.062991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:08.173949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f26408-a884-489a-8507-d98951bbd34e None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:08.188632 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:08.212058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850f28b2-2e5d-4505-a980-26047276155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:08.212673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f28b2-2e5d-4505-a980-26047276155f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:08.212949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f28b2-2e5d-4505-a980-26047276155f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:08.213218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850f28b2-2e5d-4505-a980-26047276155f None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:08.213853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f28b2-2e5d-4505-a980-26047276155f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:08.215230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850f28b2-2e5d-4505-a980-26047276155f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:08.229665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 497/989] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 13:59:08 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 13:59:08.773672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576288a-ff09-4650-b4b4-40c8494fa8d5 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:08.774238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576288a-ff09-4650-b4b4-40c8494fa8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:08.774238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0576288a-ff09-4650-b4b4-40c8494fa8d5 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:08.775656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 493/990] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:08.780866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8763-a20d-4aab-b7d6-255604d639b4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:08.781293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8763-a20d-4aab-b7d6-255604d639b4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:08.781434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8763-a20d-4aab-b7d6-255604d639b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:08.800600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-54de8763-a20d-4aab-b7d6-255604d639b4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:08.801856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-54de8763-a20d-4aab-b7d6-255604d639b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:08.869150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 498/991] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:08 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:09.128242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576288a-ff09-4650-b4b4-40c8494fa8d5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:09.213544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:09.235065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0576288a-ff09-4650-b4b4-40c8494fa8d5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:09.268259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:09.268885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:09.269157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:09.269424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:09.270026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:09.271400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:09.280009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:09.281007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:09.282381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:09.286023 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 494/992] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:09.287213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de8763-a20d-4aab-b7d6-255604d639b4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:09.386121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de8763-a20d-4aab-b7d6-255604d639b4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:09.395293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:09.422075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:09.422705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:09.422979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:09.423246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:09.423943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:09.424240 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:09.425681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:09.433711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 499/993] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:59:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:09.436390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:09.555714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d938f03-bc82-4f20-9c9c-94bc115be241 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:09.590828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:09.591448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:09.591720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:09.591989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:09.592634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:09.592927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:09.594399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:09.603870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 495/994] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:59:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:09.605519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:09.709709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34985686-8af6-4d30-967b-579ad0395f1c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:09.742920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef22125-230d-4a91-b45a-90dd115d3855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:09.743534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef22125-230d-4a91-b45a-90dd115d3855 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:09.743807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef22125-230d-4a91-b45a-90dd115d3855 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:09.744081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef22125-230d-4a91-b45a-90dd115d3855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:09.744779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef22125-230d-4a91-b45a-90dd115d3855 None admin] RBAC: Adding query filter params (interface=public, service_id=b8fe8a020ff146f781dc7921b4b3b814, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:09.745083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef22125-230d-4a91-b45a-90dd115d3855 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:09.746489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef22125-230d-4a91-b45a-90dd115d3855 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:09.751908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 500/995] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 13:59:09 2022] GET /identity/v3/endpoints?service_id=b8fe8a020ff146f781dc7921b4b3b814&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:09.757543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:09.768664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:09.866795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a96b6f-ee7b-415f-b629-14f7eca243d9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:09.902022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91770c59-99d0-4c79-9b99-dc2aa423b4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:09.902676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91770c59-99d0-4c79-9b99-dc2aa423b4f9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:09.902944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91770c59-99d0-4c79-9b99-dc2aa423b4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:09.903213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91770c59-99d0-4c79-9b99-dc2aa423b4f9 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:09.903841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91770c59-99d0-4c79-9b99-dc2aa423b4f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:09.905287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91770c59-99d0-4c79-9b99-dc2aa423b4f9 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:09.912918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 496/996] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 13:59:09 2022] GET /identity/v3/services => generated 1727 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 13:59:10.021854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b8a15-579b-4df1-80c3-9246c7d2f1d4 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:10.022371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b8a15-579b-4df1-80c3-9246c7d2f1d4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:10.022371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6b8a15-579b-4df1-80c3-9246c7d2f1d4 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:10.023737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 501/997] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:10.029520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0e1c4-9d68-4fc9-bb0b-2e0afd3e1a39 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:10.029876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0e1c4-9d68-4fc9-bb0b-2e0afd3e1a39 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:10.030060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0e1c4-9d68-4fc9-bb0b-2e0afd3e1a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:10.050061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-21e0e1c4-9d68-4fc9-bb0b-2e0afd3e1a39 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:10.051613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-21e0e1c4-9d68-4fc9-bb0b-2e0afd3e1a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:10.134603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 497/998] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:10.396186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b8a15-579b-4df1-80c3-9246c7d2f1d4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:10.396569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225db6a0-efe0-40e3-afd3-1c50b39ca8bc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:10.396871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225db6a0-efe0-40e3-afd3-1c50b39ca8bc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:10.397127 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225db6a0-efe0-40e3-afd3-1c50b39ca8bc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:10.398701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 498/999] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:10.403908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39e27b-6bae-4e9c-9f99-f6b4d35c21fc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:10.404209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39e27b-6bae-4e9c-9f99-f6b4d35c21fc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:10.404460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39e27b-6bae-4e9c-9f99-f6b4d35c21fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:10.412028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:10.433253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fe39e27b-6bae-4e9c-9f99-f6b4d35c21fc None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:10.434494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fe39e27b-6bae-4e9c-9f99-f6b4d35c21fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:10.442600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:10.521874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6b8a15-579b-4df1-80c3-9246c7d2f1d4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:10.528430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 499/1000] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:10 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:59:10.560438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:10.561071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:10.561353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:10.561609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] PATH_INFO: `/v3/services/image` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:10.562216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:10.563581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:10.572382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:10.573371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:10.574707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 502/1001] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Oct 5 13:59:10 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:10.579415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39e27b-6bae-4e9c-9f99-f6b4d35c21fc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:10.690772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39e27b-6bae-4e9c-9f99-f6b4d35c21fc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:10.723251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:10.728972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:10.729591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:10.729894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:10.730154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:10.731083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] RBAC: Adding query filter params (name=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:10.731387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:10.732924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:10.738770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 500/1002] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:10.744328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:10.813470 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:10.874367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:10.878225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0134d-95cd-42d4-8fa8-e8b6fb6627bb None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:10.917157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:10.917806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:10.918085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:10.918346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:10.918979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] RBAC: Adding query filter params (type=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:10.919278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:10.920654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:10.925790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 503/1003] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:10.926882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb9a5d-f803-4e3d-b04a-3e2b0b932c81 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:10.971094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:10.971659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:10.971936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:10.972204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:10.972800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:10.974218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:10.983844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:10.984888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 05 13:59:10.986331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:10.990094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 501/1004] 10.222.0.23 () {56 vars in 1126 bytes} [Wed Oct 5 13:59:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:10.991832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:11.109291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7775e-568b-4f7f-b486-9322faa0d740 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:11.148167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:11.149292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:11.149801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:11.150334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:11.151507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:11.152039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:11.154144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:11.160880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 504/1005] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:59:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:11.169155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:11.237313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:11.282158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d223d9-86e2-4a7a-866f-cd02740165d5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:11.318337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b101a273-2424-40e7-b2ba-5b9d51182ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:11.318938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101a273-2424-40e7-b2ba-5b9d51182ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:11.319211 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101a273-2424-40e7-b2ba-5b9d51182ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:11.319454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b101a273-2424-40e7-b2ba-5b9d51182ab8 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:11.320091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101a273-2424-40e7-b2ba-5b9d51182ab8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:11.320388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101a273-2424-40e7-b2ba-5b9d51182ab8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:11.321783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b101a273-2424-40e7-b2ba-5b9d51182ab8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:11.326863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 502/1006] 10.222.0.23 () {56 vars in 1122 bytes} [Wed Oct 5 13:59:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:11.332742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:11.362168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:11.434229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1c973-6fb8-4193-a1f8-ce4d6d502e26 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:11.469169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb603b1-fa3b-416f-a158-d9c739454daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:11.469806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb603b1-fa3b-416f-a158-d9c739454daf None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:11.470058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb603b1-fa3b-416f-a158-d9c739454daf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:11.470399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb603b1-fa3b-416f-a158-d9c739454daf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:11.471059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb603b1-fa3b-416f-a158-d9c739454daf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:11.472431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb603b1-fa3b-416f-a158-d9c739454daf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:11.497773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 505/1007] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 13:59:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:12.103889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d77755-9341-453e-a93b-fdf3ec0251b6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:12.112015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d77755-9341-453e-a93b-fdf3ec0251b6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:12.112015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d77755-9341-453e-a93b-fdf3ec0251b6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:12.112015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 503/1008] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:12.113473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24521-9def-46ff-b08b-fbd2ebde8d4f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:12.113947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24521-9def-46ff-b08b-fbd2ebde8d4f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:12.114031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b24521-9def-46ff-b08b-fbd2ebde8d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:12.132555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-80b24521-9def-46ff-b08b-fbd2ebde8d4f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:12.133789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-80b24521-9def-46ff-b08b-fbd2ebde8d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:12.208931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 506/1009] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:12 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:59:12.470100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d77755-9341-453e-a93b-fdf3ec0251b6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:12.581576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d77755-9341-453e-a93b-fdf3ec0251b6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:12.604556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:12.617077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:12.617743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:12.617999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:12.618262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:12.618837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:12.620187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:12.628960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:12.633868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:12.634845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 504/1010] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:12.636950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b24521-9def-46ff-b08b-fbd2ebde8d4f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:12.653528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:12.746216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b24521-9def-46ff-b08b-fbd2ebde8d4f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:12.780112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:12.780721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:12.780995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:12.781264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:12.781916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:12.782218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:12.783643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:12.793043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 507/1011] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:59:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:12.794417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:12.898576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e71250-1ff1-40d8-8d1c-8c1194987474 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:12.931688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e789428-6bbf-4718-aed5-797d7c89fb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:12.932290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e789428-6bbf-4718-aed5-797d7c89fb97 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:12.932561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e789428-6bbf-4718-aed5-797d7c89fb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:12.932821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e789428-6bbf-4718-aed5-797d7c89fb97 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:12.933437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e789428-6bbf-4718-aed5-797d7c89fb97 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:12.933764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e789428-6bbf-4718-aed5-797d7c89fb97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:12.935114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e789428-6bbf-4718-aed5-797d7c89fb97 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:12.940235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 505/1012] 10.222.0.23 () {56 vars in 1112 bytes} [Wed Oct 5 13:59:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:12.949754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:13.036248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:13.059522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec8ca87-937a-4079-b97d-80f2e8435318 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:13.093568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cb63f-a6ed-4d0c-83d6-64269c2bef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:13.094208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cb63f-a6ed-4d0c-83d6-64269c2bef2f None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:13.094482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cb63f-a6ed-4d0c-83d6-64269c2bef2f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:13.094746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cb63f-a6ed-4d0c-83d6-64269c2bef2f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:13.095474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cb63f-a6ed-4d0c-83d6-64269c2bef2f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:13.096852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6cb63f-a6ed-4d0c-83d6-64269c2bef2f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:13.114726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece914aa-d53d-4be6-95a6-52c32dfd27e2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:13.115268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece914aa-d53d-4be6-95a6-52c32dfd27e2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:13.115268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece914aa-d53d-4be6-95a6-52c32dfd27e2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:13.116724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 506/1013] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:13.121859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 508/1014] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 13:59:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:13.122341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00baf635-b790-4cf4-99fa-f706be253bfa None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:13.122341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00baf635-b790-4cf4-99fa-f706be253bfa None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:13.122341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00baf635-b790-4cf4-99fa-f706be253bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:13.141861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-00baf635-b790-4cf4-99fa-f706be253bfa None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:13.143120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-00baf635-b790-4cf4-99fa-f706be253bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:13.196518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:13.225701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 507/1015] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:13 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:13.484790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cb63f-a6ed-4d0c-83d6-64269c2bef2f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:13.600927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6cb63f-a6ed-4d0c-83d6-64269c2bef2f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:13.634603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef50b84-5672-4e88-847f-a082171e2d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:13.635219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef50b84-5672-4e88-847f-a082171e2d16 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:13.635493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef50b84-5672-4e88-847f-a082171e2d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:13.635832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef50b84-5672-4e88-847f-a082171e2d16 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:13.636503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef50b84-5672-4e88-847f-a082171e2d16 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:13.637908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef50b84-5672-4e88-847f-a082171e2d16 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:13.641201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:13.655610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 509/1016] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:59:13 2022] POST /identity/v3/services => generated 242 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:13.656729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1017ec-8068-44d2-9beb-5219114006f8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:13.657032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1017ec-8068-44d2-9beb-5219114006f8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:13.657304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1017ec-8068-44d2-9beb-5219114006f8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:13.658859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 508/1017] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:13.664584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921c414-7ef8-4203-9e2e-90586924aa23 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:13.664931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921c414-7ef8-4203-9e2e-90586924aa23 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:13.665100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2921c414-7ef8-4203-9e2e-90586924aa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:13.685699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2921c414-7ef8-4203-9e2e-90586924aa23 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:13.685969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2921c414-7ef8-4203-9e2e-90586924aa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:13.768080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 510/1018] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:14.029145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1017ec-8068-44d2-9beb-5219114006f8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:14.142543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1017ec-8068-44d2-9beb-5219114006f8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:14.178420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077e67e-402c-406c-85f3-770732994bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:14.179113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e67e-402c-406c-85f3-770732994bbf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:14.179392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e67e-402c-406c-85f3-770732994bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:14.179660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077e67e-402c-406c-85f3-770732994bbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:14.180763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077e67e-402c-406c-85f3-770732994bbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:14.182228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077e67e-402c-406c-85f3-770732994bbf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:14.184162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 509/1019] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:14.189836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921c414-7ef8-4203-9e2e-90586924aa23 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:14.306182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2921c414-7ef8-4203-9e2e-90586924aa23 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:14.331752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:14.348445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4dede1-61fd-41d7-8548-1ac6603a577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:14.349402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dede1-61fd-41d7-8548-1ac6603a577c None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:14.349959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dede1-61fd-41d7-8548-1ac6603a577c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:14.350447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4dede1-61fd-41d7-8548-1ac6603a577c None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:14.351306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dede1-61fd-41d7-8548-1ac6603a577c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:14.353078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4dede1-61fd-41d7-8548-1ac6603a577c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:14.398153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 511/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:59:14 2022] POST /identity/v3/projects => generated 311 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:15.327955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c3069-4fb6-421d-8529-cc9df6536a80 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:15.328532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c3069-4fb6-421d-8529-cc9df6536a80 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:15.328532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c3069-4fb6-421d-8529-cc9df6536a80 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:15.333503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 510/1021] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:15.335188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10210b89-7cc9-4396-a95f-51468fea7423 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:15.335617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10210b89-7cc9-4396-a95f-51468fea7423 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:15.335695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10210b89-7cc9-4396-a95f-51468fea7423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:15.354043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-10210b89-7cc9-4396-a95f-51468fea7423 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:15.355210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10210b89-7cc9-4396-a95f-51468fea7423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:15.425684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 512/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:15.715110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c3069-4fb6-421d-8529-cc9df6536a80 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:15.767990 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:15.822802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3c3069-4fb6-421d-8529-cc9df6536a80 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:15.825894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c469-46ec-4dc1-8dc4-a68868bc9b06 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:15.826180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c469-46ec-4dc1-8dc4-a68868bc9b06 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:15.826448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad4c469-46ec-4dc1-8dc4-a68868bc9b06 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:15.827937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 513/1023] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:15.833099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f94e2-091d-4469-9194-a9dd952b43ee None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:15.833591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f94e2-091d-4469-9194-a9dd952b43ee None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:15.834086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79f94e2-091d-4469-9194-a9dd952b43ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:15.852805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b79f94e2-091d-4469-9194-a9dd952b43ee None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:15.854275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b79f94e2-091d-4469-9194-a9dd952b43ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:15.859839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10676e0e-3213-42ab-a5ac-fa30559c54b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:15.860801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676e0e-3213-42ab-a5ac-fa30559c54b9 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:15.861081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676e0e-3213-42ab-a5ac-fa30559c54b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:15.861345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10676e0e-3213-42ab-a5ac-fa30559c54b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:15.872769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10676e0e-3213-42ab-a5ac-fa30559c54b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-10676e0e-3213-42ab-a5ac-fa30559c54b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:15.877480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:15.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:15.879267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 511/1024] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 13:59:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:15.884359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10676e0e-3213-42ab-a5ac-fa30559c54b9 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:15.924999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 514/1025] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:59:15.996371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10676e0e-3213-42ab-a5ac-fa30559c54b9 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.003961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:16.031428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.032037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.032312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.032623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.033342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:16.033657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.035014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.043594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 512/1026] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 13:59:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:16.049508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f94e2-091d-4469-9194-a9dd952b43ee None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.081557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:16.155070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79f94e2-091d-4469-9194-a9dd952b43ee None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.190762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.191401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.191676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.191949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.202259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.203708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.206112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:16.218869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:16.219861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:16.220826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:16.220826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:16.220826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:16.223607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 515/1027] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:16.225706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.316913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:16.322150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c33daa9-da59-4f3a-a02e-b29bcd460c00 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.334376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08764c79-b66c-474e-9397-436cd4c3b08a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.357739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.358459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.358740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.359009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] PATH_INFO: `/v3/services/image` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.359608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.361346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.369919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:16.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:16.371425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:16.372247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 513/1028] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:16.373797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.374208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.374620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.375390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:16.375831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.377457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.377572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.390212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 516/1029] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:16.395983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.409239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:16.482576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7cdbeb-1f82-4583-80c6-d2fce8bfb51f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.510102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e46a17b-a319-4626-b82b-52cec93756d0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.519795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.520411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.520684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.520965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.521578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] RBAC: Adding query filter params (name=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:16.521906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.523261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.528654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 514/1030] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:16.534649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.547784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad3482-e30e-447b-87a7-88210d63984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.548648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3482-e30e-447b-87a7-88210d63984e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.549080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3482-e30e-447b-87a7-88210d63984e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.549512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad3482-e30e-447b-87a7-88210d63984e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.557088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad3482-e30e-447b-87a7-88210d63984e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.558565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad3482-e30e-447b-87a7-88210d63984e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-41ad3482-e30e-447b-87a7-88210d63984e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:59:16.570193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:16.571328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:16.572281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 517/1031] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:16.572897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ccc2c-1f99-4ae0-b8fb-571a14f96c19 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.572897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ccc2c-1f99-4ae0-b8fb-571a14f96c19 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.573144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07ccc2c-1f99-4ae0-b8fb-571a14f96c19 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.574392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 518/1032] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:16.576720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ccc2c-1f99-4ae0-b8fb-571a14f96c19 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.581480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:16.631021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:16.638791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9778b864-9bc9-45f7-a5a2-b835ce2bc16a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.672454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306fdbad-de6c-42f9-9ddd-fe1e0c5e244f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.673074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fdbad-de6c-42f9-9ddd-fe1e0c5e244f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.673356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fdbad-de6c-42f9-9ddd-fe1e0c5e244f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.673625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306fdbad-de6c-42f9-9ddd-fe1e0c5e244f None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.674277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fdbad-de6c-42f9-9ddd-fe1e0c5e244f None admin] RBAC: Adding query filter params (type=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:16.674579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fdbad-de6c-42f9-9ddd-fe1e0c5e244f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.675937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306fdbad-de6c-42f9-9ddd-fe1e0c5e244f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.680539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 515/1033] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:16.681688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245dc23-f861-42e1-85ec-26f3436b86d0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.681974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245dc23-f861-42e1-85ec-26f3436b86d0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.682216 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2245dc23-f861-42e1-85ec-26f3436b86d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.684316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07ccc2c-1f99-4ae0-b8fb-571a14f96c19 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.699461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2245dc23-f861-42e1-85ec-26f3436b86d0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:16.700650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2245dc23-f861-42e1-85ec-26f3436b86d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:16.721896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.722500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.722777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.723042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.723767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:16.724067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.725491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.742306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 519/1034] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:16.743099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.772233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 516/1035] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:16.773067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245dc23-f861-42e1-85ec-26f3436b86d0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.808230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:16.857174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc22364-5e73-41d5-9c05-85ed6f5da43c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.863767 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:16.875601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2245dc23-f861-42e1-85ec-26f3436b86d0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:16.897624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.898233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.898508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.898773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.899466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] RBAC: Adding query filter params (interface=public, service_id=6591e52f36614b718004ff80abd438f8, region_id=RegionOne) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:16.899768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.901122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:16.906151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 520/1036] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/endpoints?service_id=6591e52f36614b718004ff80abd438f8&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:16.911681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:16.913198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81341d50-cd79-485c-9cdb-24521fa18904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:16.913834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81341d50-cd79-485c-9cdb-24521fa18904 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:16.914106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81341d50-cd79-485c-9cdb-24521fa18904 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:16.914478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81341d50-cd79-485c-9cdb-24521fa18904 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:16.984391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81341d50-cd79-485c-9cdb-24521fa18904 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:16.986265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81341d50-cd79-485c-9cdb-24521fa18904 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.017372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acead336-c7f0-4d3d-b556-6fc47ea98401 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:17.042653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:17.052291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c81eb-0fd1-401a-9bfa-41a03f0f5664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:17.052902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c81eb-0fd1-401a-9bfa-41a03f0f5664 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:17.053175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c81eb-0fd1-401a-9bfa-41a03f0f5664 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:17.053443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c81eb-0fd1-401a-9bfa-41a03f0f5664 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:17.054097 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c81eb-0fd1-401a-9bfa-41a03f0f5664 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:17.055454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c81eb-0fd1-401a-9bfa-41a03f0f5664 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.061743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 521/1037] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 13:59:17.062447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c81eb-0fd1-401a-9bfa-41a03f0f5664 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:17.081101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:17.092160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 517/1038] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:16 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:59:17.097997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81341d50-cd79-485c-9cdb-24521fa18904 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:17.174662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c81eb-0fd1-401a-9bfa-41a03f0f5664 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:17.212119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bc4d6-60c6-4808-ba4c-e51835655539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:17.212735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bc4d6-60c6-4808-ba4c-e51835655539 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:17.213003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bc4d6-60c6-4808-ba4c-e51835655539 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:17.213271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6bc4d6-60c6-4808-ba4c-e51835655539 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:17.214439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bc4d6-60c6-4808-ba4c-e51835655539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:17.215965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6bc4d6-60c6-4808-ba4c-e51835655539 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.217212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81341d50-cd79-485c-9cdb-24521fa18904 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:17.221668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 522/1039] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:17.224678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bc4d6-60c6-4808-ba4c-e51835655539 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:17.257415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:17.258380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:17.258650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:17.258912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:17.269182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:17.270636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:17.285434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:17.286613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:17.287857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:17.287857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:17.287857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:17.291595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 518/1040] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:59:17.292449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:17.298917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:17.336425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:17.343895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6bc4d6-60c6-4808-ba4c-e51835655539 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:17.378478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c2b8-74b2-4607-8bcf-81bf6c91a16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:17.379875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c2b8-74b2-4607-8bcf-81bf6c91a16b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:17.380372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c2b8-74b2-4607-8bcf-81bf6c91a16b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:17.380836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e6c2b8-74b2-4607-8bcf-81bf6c91a16b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:17.381756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c2b8-74b2-4607-8bcf-81bf6c91a16b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:17.383392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e6c2b8-74b2-4607-8bcf-81bf6c91a16b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.402877 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909638ab-1d75-46c2-a840-326d020e669b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:17.408217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 523/1041] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:59:17 2022] POST /identity/v3/users => generated 285 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:17.448524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:17.449062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:17.449335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:17.449601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:17.450283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:17.450629 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:17.452003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.465807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 519/1042] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:17.471234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c2b8-74b2-4607-8bcf-81bf6c91a16b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:17.592596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e6c2b8-74b2-4607-8bcf-81bf6c91a16b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:17.607185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:17.629964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:17.630584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:17.630879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:17.631127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:17.638057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:17.639507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:59:17.651311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:17.652264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:17.653793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 524/1043] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:17.658484 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:17.758474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6afd6a-5b3d-4467-808a-b03f77a3a460 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:17.759904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:17.795155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf95da6-c13e-4a81-8f20-fe6273e0ad0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:17.795767 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf95da6-c13e-4a81-8f20-fe6273e0ad0c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:17.796039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf95da6-c13e-4a81-8f20-fe6273e0ad0c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:17.796304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf95da6-c13e-4a81-8f20-fe6273e0ad0c None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:17.796996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf95da6-c13e-4a81-8f20-fe6273e0ad0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:17.797296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf95da6-c13e-4a81-8f20-fe6273e0ad0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:17.798803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf95da6-c13e-4a81-8f20-fe6273e0ad0c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.810587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 520/1044] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:59:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:17.816735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:17.915990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8f3005-703b-415d-91e1-bdf5601b1267 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:17.953231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c2a47d-f6e3-4bf5-8a79-7ce1df97cc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:17.953865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a47d-f6e3-4bf5-8a79-7ce1df97cc41 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:17.954141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a47d-f6e3-4bf5-8a79-7ce1df97cc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:17.954477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c2a47d-f6e3-4bf5-8a79-7ce1df97cc41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:17.955187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2a47d-f6e3-4bf5-8a79-7ce1df97cc41 None admin] RBAC: Adding query filter params (role.id=15185fccbb00430dac762cb64ecf6355, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=83f4897f360f4f23aac5749685fdf500) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:17.955490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2a47d-f6e3-4bf5-8a79-7ce1df97cc41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:17.956918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c2a47d-f6e3-4bf5-8a79-7ce1df97cc41 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:17.962620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 525/1045] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:17 2022] GET /identity/v3/role_assignments?user.id=83f4897f360f4f23aac5749685fdf500&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=15185fccbb00430dac762cb64ecf6355 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:19.174716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982b66d-a1b8-4413-bfee-aefa82a3cbe6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:19.175302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982b66d-a1b8-4413-bfee-aefa82a3cbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:19.175302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982b66d-a1b8-4413-bfee-aefa82a3cbe6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:19.176728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 521/1046] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:19.181881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9211ea-1a4e-4485-8ab9-e3e4f2b1f468 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:19.182327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9211ea-1a4e-4485-8ab9-e3e4f2b1f468 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:19.182417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9211ea-1a4e-4485-8ab9-e3e4f2b1f468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:19.200908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3a9211ea-1a4e-4485-8ab9-e3e4f2b1f468 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:19.202109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a9211ea-1a4e-4485-8ab9-e3e4f2b1f468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:19.233249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:19.275692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 526/1047] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:19.536382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982b66d-a1b8-4413-bfee-aefa82a3cbe6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:19.593580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae057bd-e34e-4999-bc81-71accbc9c50f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:19.594211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae057bd-e34e-4999-bc81-71accbc9c50f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:19.594662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae057bd-e34e-4999-bc81-71accbc9c50f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:19.596369 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 527/1048] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:19.601346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93d92-4e09-4982-8c14-fc8567fa4f0a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:19.601935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93d92-4e09-4982-8c14-fc8567fa4f0a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:19.602210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c93d92-4e09-4982-8c14-fc8567fa4f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:19.620446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f5c93d92-4e09-4982-8c14-fc8567fa4f0a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:19.621618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f5c93d92-4e09-4982-8c14-fc8567fa4f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:19.647819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982b66d-a1b8-4413-bfee-aefa82a3cbe6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:19.683131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:19.684103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:19.684567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:19.684950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] PATH_INFO: `/v3/services/image` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:19.685528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:19.686908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:19.693700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 528/1049] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:19.695778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:19.696737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 05 13:59:19.697559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:19.701803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 522/1050] 10.222.0.23 () {56 vars in 1105 bytes} [Wed Oct 5 13:59:19 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:19.702831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c93d92-4e09-4982-8c14-fc8567fa4f0a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:19.797344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:19.801699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c93d92-4e09-4982-8c14-fc8567fa4f0a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:19.835921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:19.836530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:19.836802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:19.837068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:19.837716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] RBAC: Adding query filter params (name=image) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:19.838013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:19.839374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:19.848837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 529/1051] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:19.849933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:19.878050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:19.951527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825468f7-6c9b-4a46-9fca-089e46010d2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:19.958424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:19.989416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:19.990052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:19.990419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:19.990628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:19.991210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] RBAC: Adding query filter params (type=image) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:19.991504 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:19.992859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:20.002751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 523/1052] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:20.003685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.067272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c4f064-42b7-4206-b286-65e95117073c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:20.097976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:20.099910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e468fe-b937-4d16-93dd-4d9c2312fbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:20.100542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e468fe-b937-4d16-93dd-4d9c2312fbcd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.100819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e468fe-b937-4d16-93dd-4d9c2312fbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.101083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e468fe-b937-4d16-93dd-4d9c2312fbcd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.107773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de47e1-7d55-40bf-98f4-b125b5614ebe None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:20.111743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e468fe-b937-4d16-93dd-4d9c2312fbcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c1e468fe-b937-4d16-93dd-4d9c2312fbcd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:20.116533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:20.117457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:20.118535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 530/1053] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:59:20.119401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236315-5133-4c44-8dac-b862e8cbb299 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.119401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236315-5133-4c44-8dac-b862e8cbb299 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.119401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47236315-5133-4c44-8dac-b862e8cbb299 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.120719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 531/1054] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:20.123540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47236315-5133-4c44-8dac-b862e8cbb299 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.143856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af71eb6-f1aa-4c1e-8c25-c55618e5333f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:20.144463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71eb6-f1aa-4c1e-8c25-c55618e5333f None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.144729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71eb6-f1aa-4c1e-8c25-c55618e5333f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.144989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af71eb6-f1aa-4c1e-8c25-c55618e5333f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.145678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af71eb6-f1aa-4c1e-8c25-c55618e5333f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:20.147033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af71eb6-f1aa-4c1e-8c25-c55618e5333f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:20.147375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:20.168570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 524/1055] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 13:59:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 13:59:20.171111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583577b-fc45-4217-9435-ecc207dd97fc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.171392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583577b-fc45-4217-9435-ecc207dd97fc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.171706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583577b-fc45-4217-9435-ecc207dd97fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.191915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6583577b-fc45-4217-9435-ecc207dd97fc None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:20.193155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6583577b-fc45-4217-9435-ecc207dd97fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:20.240051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47236315-5133-4c44-8dac-b862e8cbb299 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:20.270023 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 525/1056] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:20.278188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:20.278957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.279327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.279754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.280549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:20.280942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:20.282534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:20.291965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 532/1057] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:59:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:20.297997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583577b-fc45-4217-9435-ecc207dd97fc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.380851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:20.405678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583577b-fc45-4217-9435-ecc207dd97fc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:20.440181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd9e97-cbfb-4968-8fe2-1471ab2d750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:20.440816 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9e97-cbfb-4968-8fe2-1471ab2d750a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.441092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9e97-cbfb-4968-8fe2-1471ab2d750a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.441361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd9e97-cbfb-4968-8fe2-1471ab2d750a None admin] PATH_INFO: `/v3/users/edb832bc4f7f44a4a4b02ca1afca16c0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.456084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd9e97-cbfb-4968-8fe2-1471ab2d750a None admin] RBAC: Authorizing `identity:get_user(user_id=edb832bc4f7f44a4a4b02ca1afca16c0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:20.457519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd9e97-cbfb-4968-8fe2-1471ab2d750a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:20.459367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 526/1058] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:20 2022] GET /identity/v3/users/edb832bc4f7f44a4a4b02ca1afca16c0 => generated 285 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:20.465026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.536192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd9e97-cbfb-4968-8fe2-1471ab2d750a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.557952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:20.578345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6d135a-45c8-40aa-b72c-2314395968fd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:20.613333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3804185-40be-48dc-81cf-d7022a6aa67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:20.614199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3804185-40be-48dc-81cf-d7022a6aa67f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.614667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3804185-40be-48dc-81cf-d7022a6aa67f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.614938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3804185-40be-48dc-81cf-d7022a6aa67f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.645671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd9e97-cbfb-4968-8fe2-1471ab2d750a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:20.682799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e050bd1-7c34-4e81-849d-4f0e95d7ce28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:20.683435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050bd1-7c34-4e81-849d-4f0e95d7ce28 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.683708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050bd1-7c34-4e81-849d-4f0e95d7ce28 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.683977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050bd1-7c34-4e81-849d-4f0e95d7ce28 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.685942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:20.690408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3804185-40be-48dc-81cf-d7022a6aa67f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:20.691912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3804185-40be-48dc-81cf-d7022a6aa67f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:20.696960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e050bd1-7c34-4e81-849d-4f0e95d7ce28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-4e050bd1-7c34-4e81-849d-4f0e95d7ce28 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:20.701774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:20.702717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:20.703560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 527/1059] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 13:59:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:20.709325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e050bd1-7c34-4e81-849d-4f0e95d7ce28 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.786386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:20.809680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 533/1060] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:59:20.813968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3804185-40be-48dc-81cf-d7022a6aa67f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.825912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e050bd1-7c34-4e81-849d-4f0e95d7ce28 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:20.862653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:20.863499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.863938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.864346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.865213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:20.865672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:20.867283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:20.876925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 528/1061] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 13:59:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:20.882459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.916465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3804185-40be-48dc-81cf-d7022a6aa67f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:20.945460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:20.951929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c846ee-08d0-4b25-b8a0-44c1d8c52b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:20.952565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c846ee-08d0-4b25-b8a0-44c1d8c52b50 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:20.952848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c846ee-08d0-4b25-b8a0-44c1d8c52b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:20.953108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c846ee-08d0-4b25-b8a0-44c1d8c52b50 None admin] PATH_INFO: `/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:20.954326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c846ee-08d0-4b25-b8a0-44c1d8c52b50 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a78b76cabe424b9939fbeb9ca861e4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:20.955756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c846ee-08d0-4b25-b8a0-44c1d8c52b50 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:20.957375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 534/1062] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:20 2022] GET /identity/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:20.963308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c846ee-08d0-4b25-b8a0-44c1d8c52b50 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:20.997866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163f8367-c423-4e0f-a31a-ba57ac71e1e2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:21.012989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:21.033226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:21.033903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:21.034184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:21.034452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:21.045339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:21.046829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:21.061081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:21.062283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:21.063179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:21.063179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:21.063179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:21.063179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 529/1063] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:21.068653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:21.074084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c846ee-08d0-4b25-b8a0-44c1d8c52b50 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:21.109100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:21.109781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:21.110035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:21.110312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:21.111366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=d9a78b76cabe424b9939fbeb9ca861e4, user.id=edb832bc4f7f44a4a4b02ca1afca16c0) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:21.111671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:21.113110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:21.118764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 535/1064] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:20 2022] GET /identity/v3/role_assignments?user.id=edb832bc4f7f44a4a4b02ca1afca16c0&scope.project.id=d9a78b76cabe424b9939fbeb9ca861e4&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:21.189486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c00aed-9edc-4af8-9fa1-30e0d313340d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:21.199965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:21.229933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:21.230886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:21.231334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:21.231773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:21.232596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:21.233092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:21.234662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:21.248802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 530/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:21.254821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:21.361500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148c7b4-6459-42f0-ab10-060971f13afe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:21.374637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:21.396977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:21.397605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:21.397924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:21.398179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:21.405786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:21.407217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:59:21.418441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:21.419410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:21.420373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 536/1066] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:59:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:21.425578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:21.533855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f365c2e-aef5-4c7f-aea6-b5e9dc444179 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:21.567162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:21.567830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:21.568055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:21.568290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:21.569006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:21.569298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:21.570715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:21.588014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 531/1067] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:59:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:21.593722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:21.692396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa27ffa-0eef-4513-a2c0-d7d7beb3dac5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:21.725063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:21.725713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:21.725965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:21.726243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:21.737407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:21.738884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:21.752726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:21.753702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:21.754683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:21.754683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:21.754683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:21.758023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 537/1068] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:21.759564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:21.787858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:21.863797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ac486-28f7-4ca1-baee-65206d195edd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:21.898455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:21.899061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:21.899330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:21.899591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:21.900227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:21.900517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:21.901898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:21.913386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 532/1069] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:21.918875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:21.944927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:22.028257 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3de571-4df6-4b07-bdd4-01e4dfa212fd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:22.061652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:22.062348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:22.062622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:22.062883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:22.069872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:22.071352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:59:22.082757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:22.083751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:22.084637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:59:22.084637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:59:22.084637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:59:22.084637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:59:22.084637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:59:22.084637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:59:22.084637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:22.084637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:22.085806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 538/1070] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:22.089889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:22.177406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:22.192796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a128cc8-aad8-4d63-bd44-6fe7c7ae8bd3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:22.229193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4f70c-1d26-4b3e-ad63-206a0096a631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:22.229883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f70c-1d26-4b3e-ad63-206a0096a631 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:22.230133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f70c-1d26-4b3e-ad63-206a0096a631 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:22.230324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4f70c-1d26-4b3e-ad63-206a0096a631 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:22.231021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4f70c-1d26-4b3e-ad63-206a0096a631 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:22.231310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4f70c-1d26-4b3e-ad63-206a0096a631 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:22.232799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4f70c-1d26-4b3e-ad63-206a0096a631 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:22.251881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 533/1071] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:22.253117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:22.334854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f16-cab5-4c6c-b816-089eb03d969d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:22.335061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f16-cab5-4c6c-b816-089eb03d969d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:22.335316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a36f16-cab5-4c6c-b816-089eb03d969d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:22.336830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 534/1072] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:22.342103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd66ec-e69c-4083-8b8d-827b669895db None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:22.342386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd66ec-e69c-4083-8b8d-827b669895db None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:22.342663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd66ec-e69c-4083-8b8d-827b669895db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:22.344933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:22.361679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-52fd66ec-e69c-4083-8b8d-827b669895db None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:22.362893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-52fd66ec-e69c-4083-8b8d-827b669895db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:22.364244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f78f52-02bc-4922-b2c7-3d29a03d8620 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:22.402273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c82861-989d-436b-9334-aae1fed8fe82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:22.402945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c82861-989d-436b-9334-aae1fed8fe82 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:22.403215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c82861-989d-436b-9334-aae1fed8fe82 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:22.403475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c82861-989d-436b-9334-aae1fed8fe82 None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/83f4897f360f4f23aac5749685fdf500/roles/15185fccbb00430dac762cb64ecf6355` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:22.406413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c82861-989d-436b-9334-aae1fed8fe82 None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=83f4897f360f4f23aac5749685fdf500, role_id=15185fccbb00430dac762cb64ecf6355)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:22.408201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c82861-989d-436b-9334-aae1fed8fe82 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:22.424744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 539/1073] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:59:22 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/83f4897f360f4f23aac5749685fdf500/roles/15185fccbb00430dac762cb64ecf6355 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:59:22.438398 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 535/1074] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:59:22.706321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c82861-989d-436b-9334-aae1fed8fe82 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:22.812117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c82861-989d-436b-9334-aae1fed8fe82 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:22.845992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533bc66-5f31-4fab-853a-c69e8b004daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:22.846616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533bc66-5f31-4fab-853a-c69e8b004daf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:22.846895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533bc66-5f31-4fab-853a-c69e8b004daf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:22.847163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533bc66-5f31-4fab-853a-c69e8b004daf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:22.862364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:22.922351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533bc66-5f31-4fab-853a-c69e8b004daf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:22.924106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533bc66-5f31-4fab-853a-c69e8b004daf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:23.029377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 540/1075] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 05 13:59:23.035259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd66ec-e69c-4083-8b8d-827b669895db None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:23.112970 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:23.139661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd66ec-e69c-4083-8b8d-827b669895db None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:23.178432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce861bc4-a1a7-43fd-857f-50709cd5ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:23.179066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce861bc4-a1a7-43fd-857f-50709cd5ffb5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:23.179337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce861bc4-a1a7-43fd-857f-50709cd5ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:23.179598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce861bc4-a1a7-43fd-857f-50709cd5ffb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:23.190171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce861bc4-a1a7-43fd-857f-50709cd5ffb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:23.191631 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce861bc4-a1a7-43fd-857f-50709cd5ffb5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ce861bc4-a1a7-43fd-857f-50709cd5ffb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:23.207012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:23.207989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:23.214196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:23.214196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:23.214196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:23.214196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 536/1076] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:23.216023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533bc66-5f31-4fab-853a-c69e8b004daf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:23.253935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123b80-44cc-437b-b890-70ffdd973375 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:23.254461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123b80-44cc-437b-b890-70ffdd973375 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:23.255400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a123b80-44cc-437b-b890-70ffdd973375 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:23.256951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 537/1077] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:23.261838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8487b53-80b5-47db-8978-61ce721e6a55 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:23.262117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8487b53-80b5-47db-8978-61ce721e6a55 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:23.262380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8487b53-80b5-47db-8978-61ce721e6a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:23.266902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:23.283195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f8487b53-80b5-47db-8978-61ce721e6a55 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:23.284675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f8487b53-80b5-47db-8978-61ce721e6a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:23.328602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533bc66-5f31-4fab-853a-c69e8b004daf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:23.361316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 538/1078] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:23.363880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:23.364498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:23.364774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:23.365038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:23.365708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:23.366012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:23.367414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:23.389661 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 541/1079] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:23.630234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8487b53-80b5-47db-8978-61ce721e6a55 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:23.692406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:23.735803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8487b53-80b5-47db-8978-61ce721e6a55 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:23.769353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4a781-985d-4ac7-bd32-01fbd3c80a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:23.770007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4a781-985d-4ac7-bd32-01fbd3c80a66 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:23.770278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4a781-985d-4ac7-bd32-01fbd3c80a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:23.770541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df4a781-985d-4ac7-bd32-01fbd3c80a66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:23.780727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df4a781-985d-4ac7-bd32-01fbd3c80a66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9df4a781-985d-4ac7-bd32-01fbd3c80a66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:23.785388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:23.786370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:23.787238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 539/1080] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:59:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:23.792669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:23.893428 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3077787-d9ca-4932-bdad-214276d6ef6a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:23.895716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:23.926569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:23.927166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:23.927436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:23.927695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:23.928344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:23.928638 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:23.930020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:23.937677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 542/1081] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:59:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:23.943523 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4a781-985d-4ac7-bd32-01fbd3c80a66 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:24.049740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df4a781-985d-4ac7-bd32-01fbd3c80a66 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:24.067156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:24.084732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8f22-fd7d-4e37-9564-7bf017973b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:24.085374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8f22-fd7d-4e37-9564-7bf017973b8d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:24.085657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8f22-fd7d-4e37-9564-7bf017973b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:24.085926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a8f22-fd7d-4e37-9564-7bf017973b8d None admin] PATH_INFO: `/v3/users/edb832bc4f7f44a4a4b02ca1afca16c0` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:24.087053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a8f22-fd7d-4e37-9564-7bf017973b8d None admin] RBAC: Authorizing `identity:get_user(user_id=edb832bc4f7f44a4a4b02ca1afca16c0)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:24.088485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a8f22-fd7d-4e37-9564-7bf017973b8d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:24.090113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 540/1082] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:23 2022] GET /identity/v3/users/edb832bc4f7f44a4a4b02ca1afca16c0 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:24.096234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:24.200529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03234746-5561-4a8b-bb5e-f49c6ddcea6a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:24.233751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e249923-4f26-49a6-b807-ff74e725d162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:24.234381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e249923-4f26-49a6-b807-ff74e725d162 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:24.234656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e249923-4f26-49a6-b807-ff74e725d162 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:24.234983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e249923-4f26-49a6-b807-ff74e725d162 None admin] PATH_INFO: `/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:24.236532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e249923-4f26-49a6-b807-ff74e725d162 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a78b76cabe424b9939fbeb9ca861e4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:24.238081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e249923-4f26-49a6-b807-ff74e725d162 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:24.239673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 543/1083] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:24 2022] GET /identity/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:24.245747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8f22-fd7d-4e37-9564-7bf017973b8d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:24.345682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a8f22-fd7d-4e37-9564-7bf017973b8d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:24.381517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ffc0d-4868-4c57-8543-7cb01354f73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:24.382262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ffc0d-4868-4c57-8543-7cb01354f73b None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:24.382536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ffc0d-4868-4c57-8543-7cb01354f73b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:24.382804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ffc0d-4868-4c57-8543-7cb01354f73b None admin] PATH_INFO: `/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4/users/edb832bc4f7f44a4a4b02ca1afca16c0/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:24.384733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ffc0d-4868-4c57-8543-7cb01354f73b None admin] RBAC: Authorizing `identity:create_grant(project_id=d9a78b76cabe424b9939fbeb9ca861e4, user_id=edb832bc4f7f44a4a4b02ca1afca16c0, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:24.386231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937ffc0d-4868-4c57-8543-7cb01354f73b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:24.404168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 541/1084] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:59:24 2022] PUT /identity/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4/users/edb832bc4f7f44a4a4b02ca1afca16c0/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:59:24.636883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c877-9b91-4cd8-b977-ecf4e70d363a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:24.637443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c877-9b91-4cd8-b977-ecf4e70d363a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:24.637443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c877-9b91-4cd8-b977-ecf4e70d363a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:24.638797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 544/1085] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:24.644201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046927a-a7b4-40ce-96b3-8db3bf089570 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:24.644485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046927a-a7b4-40ce-96b3-8db3bf089570 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:24.644751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046927a-a7b4-40ce-96b3-8db3bf089570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:24.663297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e046927a-a7b4-40ce-96b3-8db3bf089570 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:24.664456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e046927a-a7b4-40ce-96b3-8db3bf089570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:24.707056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:24.732139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 542/1086] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:24.993262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c877-9b91-4cd8-b977-ecf4e70d363a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:25.031103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:25.099829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c877-9b91-4cd8-b977-ecf4e70d363a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:25.136424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68188ce1-5d81-46a4-92c0-030b555128c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:25.137088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68188ce1-5d81-46a4-92c0-030b555128c0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:25.137365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68188ce1-5d81-46a4-92c0-030b555128c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:25.137670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68188ce1-5d81-46a4-92c0-030b555128c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:25.148102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68188ce1-5d81-46a4-92c0-030b555128c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-68188ce1-5d81-46a4-92c0-030b555128c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:25.152898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:25.153956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:25.155406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 545/1087] 10.222.0.23 () {56 vars in 1120 bytes} [Wed Oct 5 13:59:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:25.160889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046927a-a7b4-40ce-96b3-8db3bf089570 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:25.264209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046927a-a7b4-40ce-96b3-8db3bf089570 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:25.299422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:25.300029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:25.300304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:25.300569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:25.301230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:25.301537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:25.302916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:25.310671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 543/1088] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 13:59:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:25.316627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68188ce1-5d81-46a4-92c0-030b555128c0 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:25.410360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:25.417453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68188ce1-5d81-46a4-92c0-030b555128c0 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:25.455243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:25.455820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:25.456099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:25.456347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:25.466358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:25.467793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:25.482351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:25.483423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:25.484451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:25.484451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:25.484451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:25.485857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 546/1089] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:25.489852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:25.532515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:25.594471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87621ce7-f275-4adb-a55c-5f87bdce8ba3 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:25.628848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:25.629443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:25.629744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:25.630004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:25.630634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:25.630930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:25.632278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:25.644443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 544/1090] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:25.650426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:25.755539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbb06a3-81e7-48c1-9f04-fff5a9979efc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:25.791612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:25.792174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:25.792447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:25.792714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:25.800774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:25.802264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 13:59:25.813407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:25.814493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:25.815446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 547/1091] 10.222.0.23 () {56 vars in 1093 bytes} [Wed Oct 5 13:59:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:25.820472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:25.917079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:25.921189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e995e9-99f7-4cc6-be69-be3a38cd7f5e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:25.955033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:25.955633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:25.955905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:25.956164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:25.956879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:25.957177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:25.958632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:25.979777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 545/1092] 10.222.0.23 () {56 vars in 1133 bytes} [Wed Oct 5 13:59:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:25.980949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:25.990982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:26.087019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90205503-f42b-404a-9537-d74789e1f2c4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:26.121883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02bf6b-a724-41cc-b180-3bec1bf94ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:26.122490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02bf6b-a724-41cc-b180-3bec1bf94ecf None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:26.122767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02bf6b-a724-41cc-b180-3bec1bf94ecf None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:26.123029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02bf6b-a724-41cc-b180-3bec1bf94ecf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:26.190298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02bf6b-a724-41cc-b180-3bec1bf94ecf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:26.191754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02bf6b-a724-41cc-b180-3bec1bf94ecf None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:26.199399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:26.290116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 548/1093] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 05 13:59:26.296059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:26.409989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89789432-dccb-4dfc-b585-1a7e0d2baaf0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:26.447313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16eaf78-368d-49c4-ba38-6c39b6209409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:26.447974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eaf78-368d-49c4-ba38-6c39b6209409 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:26.448250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eaf78-368d-49c4-ba38-6c39b6209409 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:26.448519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16eaf78-368d-49c4-ba38-6c39b6209409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:26.458921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eaf78-368d-49c4-ba38-6c39b6209409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:26.460351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16eaf78-368d-49c4-ba38-6c39b6209409 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-d16eaf78-368d-49c4-ba38-6c39b6209409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 13:59:26.474910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:26.476007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 13:59:26.482061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 13:59:26.482061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 13:59:26.482061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:26.482061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 546/1094] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:26.482360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02bf6b-a724-41cc-b180-3bec1bf94ecf None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:26.507293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030a49-eb7e-47ee-afc2-98f8933c9dcd None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:26.507817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030a49-eb7e-47ee-afc2-98f8933c9dcd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:26.507817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12030a49-eb7e-47ee-afc2-98f8933c9dcd None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:26.509392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 547/1095] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 13:59:26.514396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab506e-349b-4246-a843-e68bbff9bb48 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:26.514670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab506e-349b-4246-a843-e68bbff9bb48 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:26.514938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab506e-349b-4246-a843-e68bbff9bb48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:26.532802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-49ab506e-349b-4246-a843-e68bbff9bb48 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:26.533989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-49ab506e-349b-4246-a843-e68bbff9bb48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:26.535925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:26.584890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02bf6b-a724-41cc-b180-3bec1bf94ecf None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:26.592490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:26.603745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 548/1096] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 13:59:26.619405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:26.620014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:26.620288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:26.620553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:26.621192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:26.621491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:26.622876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:26.633592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 549/1097] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 13:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:26.639402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab506e-349b-4246-a843-e68bbff9bb48 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:26.743969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab506e-349b-4246-a843-e68bbff9bb48 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:26.779650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:26.780292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:26.780559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:26.780820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:26.789802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:26.791296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:26.793501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 13:59:26.804536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:26.805509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:26.806984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 549/1098] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 13:59:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:26.811738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:26.890771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:26.912849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19b8e1-49ab-4258-89bf-ab6a92628736 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:26.947357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:26.947985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:26.948258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:26.948522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:26.949214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:26.949529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:26.951001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:26.963125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 550/1099] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 13:59:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:26.969199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:26.989863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccde0f0-4295-448d-a7cd-e27705dcec7c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:26.996516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:27.023687 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46fa4b-3981-4b22-8c6f-cb31aa89df1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:27.024336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46fa4b-3981-4b22-8c6f-cb31aa89df1f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:27.024615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46fa4b-3981-4b22-8c6f-cb31aa89df1f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:27.024881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c46fa4b-3981-4b22-8c6f-cb31aa89df1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:27.035668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c46fa4b-3981-4b22-8c6f-cb31aa89df1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5c46fa4b-3981-4b22-8c6f-cb31aa89df1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:27.040308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:27.041389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:27.042289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 550/1100] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:59:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 13:59:27.049194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46fa4b-3981-4b22-8c6f-cb31aa89df1f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:27.071886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df69cf69-8bd3-4e21-8371-af7f6208099b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:27.105574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:27.106200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:27.106470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:27.106734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:27.107447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] RBAC: Adding query filter params (role.id=15185fccbb00430dac762cb64ecf6355, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=83f4897f360f4f23aac5749685fdf500) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:27.107748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:27.109186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:27.110705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:27.115847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 551/1101] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:26 2022] GET /identity/v3/role_assignments?user.id=83f4897f360f4f23aac5749685fdf500&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=15185fccbb00430dac762cb64ecf6355 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:27.158017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c46fa4b-3981-4b22-8c6f-cb31aa89df1f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:27.195924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:27.196542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:27.196817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:27.197089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:27.197810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:27.198113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:27.199497 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:27.209407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 551/1102] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:59:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:27.215581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:27.312461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:27.322737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dcc663-9fd8-4093-bc31-97f0c62a7dae None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:27.356633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca1ac3-f212-4ebc-a15e-90b0927b7750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:27.357268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca1ac3-f212-4ebc-a15e-90b0927b7750 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:27.357545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca1ac3-f212-4ebc-a15e-90b0927b7750 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:27.357841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ca1ac3-f212-4ebc-a15e-90b0927b7750 None admin] PATH_INFO: `/v3/users/edb832bc4f7f44a4a4b02ca1afca16c0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:27.358969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca1ac3-f212-4ebc-a15e-90b0927b7750 None admin] RBAC: Authorizing `identity:get_user(user_id=edb832bc4f7f44a4a4b02ca1afca16c0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:27.360715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ca1ac3-f212-4ebc-a15e-90b0927b7750 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:27.362326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 552/1103] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:27 2022] GET /identity/v3/users/edb832bc4f7f44a4a4b02ca1afca16c0 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:27.368238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:27.467864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2857210-8ccd-4b30-bbc8-2cf4a233cec6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:27.488032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:27.502118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda45f5-99ff-4f5d-b4ca-cc88ba1726dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:27.502744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda45f5-99ff-4f5d-b4ca-cc88ba1726dc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:27.503017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda45f5-99ff-4f5d-b4ca-cc88ba1726dc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:27.503288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cda45f5-99ff-4f5d-b4ca-cc88ba1726dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:27.572552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda45f5-99ff-4f5d-b4ca-cc88ba1726dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:27.573973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cda45f5-99ff-4f5d-b4ca-cc88ba1726dc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:27.669452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 552/1104] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:27 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:59:27.675728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca1ac3-f212-4ebc-a15e-90b0927b7750 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:27.778142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ca1ac3-f212-4ebc-a15e-90b0927b7750 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:27.811192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce884cbe-5393-431a-b5fe-ab78fb2fae2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:27.811835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce884cbe-5393-431a-b5fe-ab78fb2fae2a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:27.812367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce884cbe-5393-431a-b5fe-ab78fb2fae2a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:27.812367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce884cbe-5393-431a-b5fe-ab78fb2fae2a None admin] PATH_INFO: `/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:27.813618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce884cbe-5393-431a-b5fe-ab78fb2fae2a None admin] RBAC: Authorizing `identity:get_project(project_id=d9a78b76cabe424b9939fbeb9ca861e4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:27.815094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce884cbe-5393-431a-b5fe-ab78fb2fae2a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:27.816884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 553/1105] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:27 2022] GET /identity/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:27.823003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda45f5-99ff-4f5d-b4ca-cc88ba1726dc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:27.843927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:27.921144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cda45f5-99ff-4f5d-b4ca-cc88ba1726dc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:27.954812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff43b9f-ed30-4559-9d0a-27ead94b6917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:27.955411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43b9f-ed30-4559-9d0a-27ead94b6917 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:27.955680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43b9f-ed30-4559-9d0a-27ead94b6917 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:27.955943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff43b9f-ed30-4559-9d0a-27ead94b6917 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:27.956674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff43b9f-ed30-4559-9d0a-27ead94b6917 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=d9a78b76cabe424b9939fbeb9ca861e4, user.id=edb832bc4f7f44a4a4b02ca1afca16c0) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:27.956974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff43b9f-ed30-4559-9d0a-27ead94b6917 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:27.958469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff43b9f-ed30-4559-9d0a-27ead94b6917 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:27.964240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 553/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:27 2022] GET /identity/v3/role_assignments?user.id=edb832bc4f7f44a4a4b02ca1afca16c0&scope.project.id=d9a78b76cabe424b9939fbeb9ca861e4&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:30.083220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb7e4d-adbe-4039-9501-efbd1e7341b0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:30.083774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb7e4d-adbe-4039-9501-efbd1e7341b0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:30.083774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb7e4d-adbe-4039-9501-efbd1e7341b0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:30.086103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 554/1107] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:30.092363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb700-3327-4018-b8cf-ed841c41c501 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:30.093126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb700-3327-4018-b8cf-ed841c41c501 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:30.093274 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb700-3327-4018-b8cf-ed841c41c501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:30.113675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3bbbb700-3327-4018-b8cf-ed841c41c501 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:30.114921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3bbbb700-3327-4018-b8cf-ed841c41c501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:30.190251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 554/1108] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:30.453534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb7e4d-adbe-4039-9501-efbd1e7341b0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:30.497708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:30.579109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb7e4d-adbe-4039-9501-efbd1e7341b0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:30.614320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f79539-a49a-44b7-8c81-6954ade349e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:30.614943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f79539-a49a-44b7-8c81-6954ade349e4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:30.615217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f79539-a49a-44b7-8c81-6954ade349e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:30.615483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f79539-a49a-44b7-8c81-6954ade349e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:30.616651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f79539-a49a-44b7-8c81-6954ade349e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:30.618118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f79539-a49a-44b7-8c81-6954ade349e4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:30.619686 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 555/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:30.628538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbb700-3327-4018-b8cf-ed841c41c501 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:30.646058 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:30.740292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbb700-3327-4018-b8cf-ed841c41c501 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:30.782209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3178420d-9385-4c89-9b20-0e75f02c469d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:30.784716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178420d-9385-4c89-9b20-0e75f02c469d None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:30.784998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178420d-9385-4c89-9b20-0e75f02c469d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:30.785269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3178420d-9385-4c89-9b20-0e75f02c469d None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:30.786225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178420d-9385-4c89-9b20-0e75f02c469d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:30.788834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3178420d-9385-4c89-9b20-0e75f02c469d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:30.814595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 555/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:59:30 2022] POST /identity/v3/users => generated 286 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:32.971026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6d3c-e0fa-40b5-8aff-e3e0efd9206d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:32.971910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6d3c-e0fa-40b5-8aff-e3e0efd9206d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:32.971910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b6d3c-e0fa-40b5-8aff-e3e0efd9206d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:32.974103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 556/1111] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:32.980336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a978ce-31b3-4248-9d5d-303231be416e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:32.980767 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a978ce-31b3-4248-9d5d-303231be416e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:32.980905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a978ce-31b3-4248-9d5d-303231be416e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:33.003258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-71a978ce-31b3-4248-9d5d-303231be416e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:33.004451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-71a978ce-31b3-4248-9d5d-303231be416e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:33.070997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:33.076629 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 556/1112] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:33.351521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6d3c-e0fa-40b5-8aff-e3e0efd9206d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:33.459721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:33.467534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b6d3c-e0fa-40b5-8aff-e3e0efd9206d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:33.504213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cfa0d-3d26-4d3e-9136-d992858a7930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:33.504859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cfa0d-3d26-4d3e-9136-d992858a7930 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:33.505135 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cfa0d-3d26-4d3e-9136-d992858a7930 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:33.505402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cfa0d-3d26-4d3e-9136-d992858a7930 None admin] PATH_INFO: `/v3/roles/5491e6c9891843b8979a054e900d2e34` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:33.506940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cfa0d-3d26-4d3e-9136-d992858a7930 None admin] RBAC: Authorizing `identity:get_role(role_id=5491e6c9891843b8979a054e900d2e34)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:33.508321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cfa0d-3d26-4d3e-9136-d992858a7930 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:33.510174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 557/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:33 2022] GET /identity/v3/roles/5491e6c9891843b8979a054e900d2e34 => generated 223 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:33.516081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a978ce-31b3-4248-9d5d-303231be416e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:33.615541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a978ce-31b3-4248-9d5d-303231be416e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:33.650705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b361d-280d-467c-845f-24126f20b1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:33.651315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b361d-280d-467c-845f-24126f20b1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:33.651584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b361d-280d-467c-845f-24126f20b1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:33.651844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b361d-280d-467c-845f-24126f20b1d4 None admin] PATH_INFO: `/v3/users/c8b7799932464d13b18e7d38e7e92596` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:33.664966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b361d-280d-467c-845f-24126f20b1d4 None admin] RBAC: Authorizing `identity:get_user(user_id=c8b7799932464d13b18e7d38e7e92596)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:33.666449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b361d-280d-467c-845f-24126f20b1d4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:33.668008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 557/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:33 2022] GET /identity/v3/users/c8b7799932464d13b18e7d38e7e92596 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:33.674945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cfa0d-3d26-4d3e-9136-d992858a7930 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:33.790787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cfa0d-3d26-4d3e-9136-d992858a7930 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:33.824921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43709d-cdde-460b-be74-801a6e3e47ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:33.825520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43709d-cdde-460b-be74-801a6e3e47ea None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:33.825827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43709d-cdde-460b-be74-801a6e3e47ea None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:33.826102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e43709d-cdde-460b-be74-801a6e3e47ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:33.857017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:33.897572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43709d-cdde-460b-be74-801a6e3e47ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:33.898987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e43709d-cdde-460b-be74-801a6e3e47ea None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:33.998152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 558/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:59:34.007645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b361d-280d-467c-845f-24126f20b1d4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:34.070388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:34.114231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b361d-280d-467c-845f-24126f20b1d4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:34.161451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772c5f5-9fbe-4a26-9638-79c266951619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:34.162113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772c5f5-9fbe-4a26-9638-79c266951619 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:34.162390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772c5f5-9fbe-4a26-9638-79c266951619 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:34.162661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772c5f5-9fbe-4a26-9638-79c266951619 None admin] PATH_INFO: `/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:34.163811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772c5f5-9fbe-4a26-9638-79c266951619 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a78b76cabe424b9939fbeb9ca861e4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:34.165240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772c5f5-9fbe-4a26-9638-79c266951619 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:34.166924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 558/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:34 2022] GET /identity/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4 => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:34.173649 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43709d-cdde-460b-be74-801a6e3e47ea None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:34.236199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:34.275504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e43709d-cdde-460b-be74-801a6e3e47ea None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:34.318914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92868878-2653-414a-95b9-71bc287052da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:34.319787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92868878-2653-414a-95b9-71bc287052da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:34.320148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92868878-2653-414a-95b9-71bc287052da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:34.320497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92868878-2653-414a-95b9-71bc287052da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:34.321593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92868878-2653-414a-95b9-71bc287052da None admin] RBAC: Adding query filter params (role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=d9a78b76cabe424b9939fbeb9ca861e4, user.id=c8b7799932464d13b18e7d38e7e92596) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:34.322085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92868878-2653-414a-95b9-71bc287052da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:34.324221 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92868878-2653-414a-95b9-71bc287052da None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:34.331515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 559/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:34 2022] GET /identity/v3/role_assignments?user.id=c8b7799932464d13b18e7d38e7e92596&scope.project.id=d9a78b76cabe424b9939fbeb9ca861e4&role.id=5491e6c9891843b8979a054e900d2e34 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:36.436818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9893b7-fdaf-45be-98f2-aa73c4f30413 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:36.437712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9893b7-fdaf-45be-98f2-aa73c4f30413 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:36.437712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9893b7-fdaf-45be-98f2-aa73c4f30413 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:36.439910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 559/1118] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:36.445346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7df3e-a0ef-4dab-bc5f-49a93e05450b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:36.445789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7df3e-a0ef-4dab-bc5f-49a93e05450b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:36.445876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7df3e-a0ef-4dab-bc5f-49a93e05450b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:36.468331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-08f7df3e-a0ef-4dab-bc5f-49a93e05450b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:36.470310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-08f7df3e-a0ef-4dab-bc5f-49a93e05450b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:36.543801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 560/1119] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:36.812585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9893b7-fdaf-45be-98f2-aa73c4f30413 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:36.945454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9893b7-fdaf-45be-98f2-aa73c4f30413 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:36.971488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:36.980708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21df21-7f10-476c-b1c2-95b6e1e5d0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:36.981346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21df21-7f10-476c-b1c2-95b6e1e5d0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:36.981620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21df21-7f10-476c-b1c2-95b6e1e5d0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:36.981913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21df21-7f10-476c-b1c2-95b6e1e5d0e7 None admin] PATH_INFO: `/v3/roles/5491e6c9891843b8979a054e900d2e34` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:36.983336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21df21-7f10-476c-b1c2-95b6e1e5d0e7 None admin] RBAC: Authorizing `identity:get_role(role_id=5491e6c9891843b8979a054e900d2e34)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:36.984738 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db21df21-7f10-476c-b1c2-95b6e1e5d0e7 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:36.986513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 560/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:36 2022] GET /identity/v3/roles/5491e6c9891843b8979a054e900d2e34 => generated 223 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:36.995913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7df3e-a0ef-4dab-bc5f-49a93e05450b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:37.055529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:37.102679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7df3e-a0ef-4dab-bc5f-49a93e05450b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:37.137627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628add-d74b-4d32-a3b2-fc120738b93f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:37.138266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628add-d74b-4d32-a3b2-fc120738b93f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:37.138542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628add-d74b-4d32-a3b2-fc120738b93f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:37.138808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5628add-d74b-4d32-a3b2-fc120738b93f None admin] PATH_INFO: `/v3/users/c8b7799932464d13b18e7d38e7e92596` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:37.139968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628add-d74b-4d32-a3b2-fc120738b93f None admin] RBAC: Authorizing `identity:get_user(user_id=c8b7799932464d13b18e7d38e7e92596)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:37.141494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5628add-d74b-4d32-a3b2-fc120738b93f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:37.143062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 561/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:36 2022] GET /identity/v3/users/c8b7799932464d13b18e7d38e7e92596 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:37.149126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21df21-7f10-476c-b1c2-95b6e1e5d0e7 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:37.251361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21df21-7f10-476c-b1c2-95b6e1e5d0e7 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:37.292539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09517f-8021-4d1b-97e5-30d77a1cadd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:37.293418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09517f-8021-4d1b-97e5-30d77a1cadd3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:37.293939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09517f-8021-4d1b-97e5-30d77a1cadd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:37.294297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09517f-8021-4d1b-97e5-30d77a1cadd3 None admin] PATH_INFO: `/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:37.296224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09517f-8021-4d1b-97e5-30d77a1cadd3 None admin] RBAC: Authorizing `identity:get_project(project_id=d9a78b76cabe424b9939fbeb9ca861e4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:37.298203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09517f-8021-4d1b-97e5-30d77a1cadd3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:37.299867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 561/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:37 2022] GET /identity/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:37.306199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628add-d74b-4d32-a3b2-fc120738b93f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:37.410206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5628add-d74b-4d32-a3b2-fc120738b93f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:37.434956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:37.448793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a6af4-a47c-4298-8363-bc5ef4d54608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:37.449438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a6af4-a47c-4298-8363-bc5ef4d54608 None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:37.449735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a6af4-a47c-4298-8363-bc5ef4d54608 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:37.450003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a6af4-a47c-4298-8363-bc5ef4d54608 None admin] PATH_INFO: `/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4/users/c8b7799932464d13b18e7d38e7e92596/roles/5491e6c9891843b8979a054e900d2e34` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:37.452515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a6af4-a47c-4298-8363-bc5ef4d54608 None admin] RBAC: Authorizing `identity:create_grant(project_id=d9a78b76cabe424b9939fbeb9ca861e4, user_id=c8b7799932464d13b18e7d38e7e92596, role_id=5491e6c9891843b8979a054e900d2e34)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:37.454004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a6af4-a47c-4298-8363-bc5ef4d54608 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:37.468087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 562/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:59:37 2022] PUT /identity/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4/users/c8b7799932464d13b18e7d38e7e92596/roles/5491e6c9891843b8979a054e900d2e34 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:59:39.574759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c3662-3fe7-4e43-ba78-7540923edce2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:39.575641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c3662-3fe7-4e43-ba78-7540923edce2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:39.575641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c3662-3fe7-4e43-ba78-7540923edce2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:39.577704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 562/1124] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:39.583913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022492-5f3d-4678-b270-51741b423fcc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:39.584620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022492-5f3d-4678-b270-51741b423fcc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:39.584732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2022492-5f3d-4678-b270-51741b423fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:39.613120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e2022492-5f3d-4678-b270-51741b423fcc None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:39.614337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e2022492-5f3d-4678-b270-51741b423fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:39.684601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 563/1125] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:39.948079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c3662-3fe7-4e43-ba78-7540923edce2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:40.028078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:40.073137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c3662-3fe7-4e43-ba78-7540923edce2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:40.108461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc633-f9e2-4d74-8333-a3f3e4427800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:40.109078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc633-f9e2-4d74-8333-a3f3e4427800 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:40.109351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc633-f9e2-4d74-8333-a3f3e4427800 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:40.109679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cc633-f9e2-4d74-8333-a3f3e4427800 None admin] PATH_INFO: `/v3/roles/5491e6c9891843b8979a054e900d2e34` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:40.111525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cc633-f9e2-4d74-8333-a3f3e4427800 None admin] RBAC: Authorizing `identity:get_role(role_id=5491e6c9891843b8979a054e900d2e34)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:40.112921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cc633-f9e2-4d74-8333-a3f3e4427800 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:40.114300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 563/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:39 2022] GET /identity/v3/roles/5491e6c9891843b8979a054e900d2e34 => generated 223 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:40.121058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2022492-5f3d-4678-b270-51741b423fcc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:40.232460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2022492-5f3d-4678-b270-51741b423fcc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:40.246805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:40.274693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3b463-03c8-4db8-9bc8-27b90bef1e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:40.275322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b463-03c8-4db8-9bc8-27b90bef1e26 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:40.275595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b463-03c8-4db8-9bc8-27b90bef1e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:40.275862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3b463-03c8-4db8-9bc8-27b90bef1e26 None admin] PATH_INFO: `/v3/users/c8b7799932464d13b18e7d38e7e92596` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:40.277044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3b463-03c8-4db8-9bc8-27b90bef1e26 None admin] RBAC: Authorizing `identity:get_user(user_id=c8b7799932464d13b18e7d38e7e92596)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:40.278516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3b463-03c8-4db8-9bc8-27b90bef1e26 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:40.280062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 564/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:40 2022] GET /identity/v3/users/c8b7799932464d13b18e7d38e7e92596 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:40.286188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc633-f9e2-4d74-8333-a3f3e4427800 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:40.394508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cc633-f9e2-4d74-8333-a3f3e4427800 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:40.427041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:40.434737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392621c8-c588-4274-b9dc-f41e12f74ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:40.435345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392621c8-c588-4274-b9dc-f41e12f74ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:40.435604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392621c8-c588-4274-b9dc-f41e12f74ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:40.435886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392621c8-c588-4274-b9dc-f41e12f74ac0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:40.505442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392621c8-c588-4274-b9dc-f41e12f74ac0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:40.506871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392621c8-c588-4274-b9dc-f41e12f74ac0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:40.609310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 564/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:59:40.616013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3b463-03c8-4db8-9bc8-27b90bef1e26 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:40.721831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3b463-03c8-4db8-9bc8-27b90bef1e26 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:40.759772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e2573-3fd9-48f8-8c29-6a07a3e9fd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:40.760323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e2573-3fd9-48f8-8c29-6a07a3e9fd2b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:40.760597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e2573-3fd9-48f8-8c29-6a07a3e9fd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:40.760866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e2573-3fd9-48f8-8c29-6a07a3e9fd2b None admin] PATH_INFO: `/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:40.762122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e2573-3fd9-48f8-8c29-6a07a3e9fd2b None admin] RBAC: Authorizing `identity:get_project(project_id=d9a78b76cabe424b9939fbeb9ca861e4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:40.763562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e2573-3fd9-48f8-8c29-6a07a3e9fd2b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:40.765155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 565/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:40 2022] GET /identity/v3/projects/d9a78b76cabe424b9939fbeb9ca861e4 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:40.772186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392621c8-c588-4274-b9dc-f41e12f74ac0 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:40.807562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:40.888476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392621c8-c588-4274-b9dc-f41e12f74ac0 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:40.927795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343318b-8260-42d3-92a9-6336684b131b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:40.928383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343318b-8260-42d3-92a9-6336684b131b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:40.928681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343318b-8260-42d3-92a9-6336684b131b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:40.928928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343318b-8260-42d3-92a9-6336684b131b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:40.929669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343318b-8260-42d3-92a9-6336684b131b None admin] RBAC: Adding query filter params (role.id=5491e6c9891843b8979a054e900d2e34, scope.project.id=d9a78b76cabe424b9939fbeb9ca861e4, user.id=c8b7799932464d13b18e7d38e7e92596) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:40.929980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343318b-8260-42d3-92a9-6336684b131b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:40.931411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343318b-8260-42d3-92a9-6336684b131b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:40.937046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 565/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:40 2022] GET /identity/v3/role_assignments?user.id=c8b7799932464d13b18e7d38e7e92596&scope.project.id=d9a78b76cabe424b9939fbeb9ca861e4&role.id=5491e6c9891843b8979a054e900d2e34 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:43.112540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee223a1-5887-4a80-9abe-218fc0388ace None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:43.113100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee223a1-5887-4a80-9abe-218fc0388ace None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:43.113100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee223a1-5887-4a80-9abe-218fc0388ace None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:43.114438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 566/1131] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:43.120895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dfe2c-d7a5-4f4b-a7a5-350a39790568 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:43.121352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dfe2c-d7a5-4f4b-a7a5-350a39790568 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:43.121457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dfe2c-d7a5-4f4b-a7a5-350a39790568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:43.143965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-301dfe2c-d7a5-4f4b-a7a5-350a39790568 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:43.145150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-301dfe2c-d7a5-4f4b-a7a5-350a39790568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:43.216954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 566/1132] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:43.486309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee223a1-5887-4a80-9abe-218fc0388ace None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:43.550104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:43.603630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee223a1-5887-4a80-9abe-218fc0388ace None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:43.639569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bddcb9b-9319-447f-9db5-4029de68ad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:43.640197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddcb9b-9319-447f-9db5-4029de68ad87 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:43.640475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddcb9b-9319-447f-9db5-4029de68ad87 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:43.640766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bddcb9b-9319-447f-9db5-4029de68ad87 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:43.642021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddcb9b-9319-447f-9db5-4029de68ad87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:43.643453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bddcb9b-9319-447f-9db5-4029de68ad87 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:43.645018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 567/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:43.651301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dfe2c-d7a5-4f4b-a7a5-350a39790568 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:43.677012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:43.756894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dfe2c-d7a5-4f4b-a7a5-350a39790568 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:43.791290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f400a-6a1a-42ac-917d-929fd75ef2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:43.791886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f400a-6a1a-42ac-917d-929fd75ef2d3 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:43.792159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f400a-6a1a-42ac-917d-929fd75ef2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:43.792425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f400a-6a1a-42ac-917d-929fd75ef2d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:43.793070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f400a-6a1a-42ac-917d-929fd75ef2d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:43.794533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130f400a-6a1a-42ac-917d-929fd75ef2d3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:43.826662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 567/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 13:59:43 2022] POST /identity/v3/projects => generated 311 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:45.974186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9ef3-8e62-4805-9cd7-f518fba527ae None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:45.975059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9ef3-8e62-4805-9cd7-f518fba527ae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:45.975059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9ef3-8e62-4805-9cd7-f518fba527ae None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:45.977194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 568/1135] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:45.983570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeb630-a8b8-4069-aaa7-7866f28dc0a5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:45.984237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeb630-a8b8-4069-aaa7-7866f28dc0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:45.984472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeb630-a8b8-4069-aaa7-7866f28dc0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:46.010061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-baeeb630-a8b8-4069-aaa7-7866f28dc0a5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:46.011255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-baeeb630-a8b8-4069-aaa7-7866f28dc0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:46.078824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 568/1136] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:46.352036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f9ef3-8e62-4805-9cd7-f518fba527ae None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:46.471026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f9ef3-8e62-4805-9cd7-f518fba527ae None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:46.488833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:46.510730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcf7211-7db4-4f93-a14c-09a97a3cbc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:46.511378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf7211-7db4-4f93-a14c-09a97a3cbc3e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:46.511652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf7211-7db4-4f93-a14c-09a97a3cbc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:46.511915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf7211-7db4-4f93-a14c-09a97a3cbc3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:46.513244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf7211-7db4-4f93-a14c-09a97a3cbc3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:46.514822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bcf7211-7db4-4f93-a14c-09a97a3cbc3e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:46.516423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 569/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 13:59:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:46.522389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeb630-a8b8-4069-aaa7-7866f28dc0a5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:46.530769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:46.627755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeb630-a8b8-4069-aaa7-7866f28dc0a5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:46.663200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840d87e1-6190-4041-94cd-ba776419936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:46.663796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d87e1-6190-4041-94cd-ba776419936b None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:46.664059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d87e1-6190-4041-94cd-ba776419936b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:46.664321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840d87e1-6190-4041-94cd-ba776419936b None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:46.664962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d87e1-6190-4041-94cd-ba776419936b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:46.666414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840d87e1-6190-4041-94cd-ba776419936b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:46.690636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 569/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 13:59:46 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 13:59:48.855191 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc1be-f3b2-42d4-99ba-fcf00a748ae5 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:48.855707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc1be-f3b2-42d4-99ba-fcf00a748ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:48.855707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66fc1be-f3b2-42d4-99ba-fcf00a748ae5 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:48.857809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 570/1139] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:48.867097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fab9db-d6c7-4ec1-8be1-c5f3ac0ee58d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:48.867497 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fab9db-d6c7-4ec1-8be1-c5f3ac0ee58d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:48.867712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fab9db-d6c7-4ec1-8be1-c5f3ac0ee58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:48.891700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-74fab9db-d6c7-4ec1-8be1-c5f3ac0ee58d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:48.892930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-74fab9db-d6c7-4ec1-8be1-c5f3ac0ee58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:48.957064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:48.970914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 570/1140] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:49.236910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fc1be-f3b2-42d4-99ba-fcf00a748ae5 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:49.353459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66fc1be-f3b2-42d4-99ba-fcf00a748ae5 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:49.394801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25e386-727e-44cc-8fc2-507511a94707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:49.395429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e386-727e-44cc-8fc2-507511a94707 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:49.395702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e386-727e-44cc-8fc2-507511a94707 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:49.395970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25e386-727e-44cc-8fc2-507511a94707 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:49.407337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25e386-727e-44cc-8fc2-507511a94707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0f25e386-727e-44cc-8fc2-507511a94707 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:49.412055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:49.413108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:49.414125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 571/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:59:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:49.419453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fab9db-d6c7-4ec1-8be1-c5f3ac0ee58d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:49.523001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fab9db-d6c7-4ec1-8be1-c5f3ac0ee58d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:49.556398 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:49.556998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:49.557272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:49.557533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:49.558218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:49.558519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:49.559866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:49.568878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 571/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:59:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:49.575296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25e386-727e-44cc-8fc2-507511a94707 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:49.627743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:49.692891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25e386-727e-44cc-8fc2-507511a94707 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:49.727520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadca58-5e2a-4430-96cb-40ceadfdf54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:49.728129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadca58-5e2a-4430-96cb-40ceadfdf54a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:49.728401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadca58-5e2a-4430-96cb-40ceadfdf54a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:49.728664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadca58-5e2a-4430-96cb-40ceadfdf54a None admin] PATH_INFO: `/v3/users/d5b399661e6441ea813446f1372fa17e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:49.742202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadca58-5e2a-4430-96cb-40ceadfdf54a None admin] RBAC: Authorizing `identity:get_user(user_id=d5b399661e6441ea813446f1372fa17e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:49.743653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadca58-5e2a-4430-96cb-40ceadfdf54a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:49.745255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 572/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:49 2022] GET /identity/v3/users/d5b399661e6441ea813446f1372fa17e => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:49.753380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:49.807459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:49.864712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c7ea2a-19b2-436c-8edc-3af053deeb2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:49.903570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd12b7-2316-4630-9185-fd1ba1a195b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:49.904231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd12b7-2316-4630-9185-fd1ba1a195b5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:49.904429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd12b7-2316-4630-9185-fd1ba1a195b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:49.904697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd12b7-2316-4630-9185-fd1ba1a195b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:49.973235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd12b7-2316-4630-9185-fd1ba1a195b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:49.974706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd12b7-2316-4630-9185-fd1ba1a195b5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:50.028758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:50.083622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 572/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:59:50.089809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadca58-5e2a-4430-96cb-40ceadfdf54a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:50.205664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadca58-5e2a-4430-96cb-40ceadfdf54a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:50.208003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:50.241435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee731aa3-2012-42d9-9fed-67b468da2c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:50.242084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731aa3-2012-42d9-9fed-67b468da2c5d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:50.242351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731aa3-2012-42d9-9fed-67b468da2c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:50.242616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee731aa3-2012-42d9-9fed-67b468da2c5d None admin] PATH_INFO: `/v3/projects/b2c92895df174b3485ed3c80bd929970` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:50.243765 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee731aa3-2012-42d9-9fed-67b468da2c5d None admin] RBAC: Authorizing `identity:get_project(project_id=b2c92895df174b3485ed3c80bd929970)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:50.245194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee731aa3-2012-42d9-9fed-67b468da2c5d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:50.246825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 573/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:50 2022] GET /identity/v3/projects/b2c92895df174b3485ed3c80bd929970 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:50.253496 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd12b7-2316-4630-9185-fd1ba1a195b5 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:50.354238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd12b7-2316-4630-9185-fd1ba1a195b5 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:50.390852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18c8055-fb1c-421b-8395-b4c050889bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:50.391492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c8055-fb1c-421b-8395-b4c050889bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:50.391756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c8055-fb1c-421b-8395-b4c050889bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:50.392022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18c8055-fb1c-421b-8395-b4c050889bb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:50.392732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c8055-fb1c-421b-8395-b4c050889bb3 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=b2c92895df174b3485ed3c80bd929970, user.id=d5b399661e6441ea813446f1372fa17e) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:50.393031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c8055-fb1c-421b-8395-b4c050889bb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:50.394464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18c8055-fb1c-421b-8395-b4c050889bb3 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:50.400328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 573/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:50 2022] GET /identity/v3/role_assignments?user.id=d5b399661e6441ea813446f1372fa17e&scope.project.id=b2c92895df174b3485ed3c80bd929970&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 13:59:52.489064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fef5da-127e-4d9d-9d8c-4f88d7c35736 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:52.490545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fef5da-127e-4d9d-9d8c-4f88d7c35736 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:52.490996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fef5da-127e-4d9d-9d8c-4f88d7c35736 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:52.493134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 574/1147] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:52.501957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3cc73-6eb8-43cc-9165-358872497f53 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:52.502690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3cc73-6eb8-43cc-9165-358872497f53 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:52.502836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb3cc73-6eb8-43cc-9165-358872497f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:52.526450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-aeb3cc73-6eb8-43cc-9165-358872497f53 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:52.527624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aeb3cc73-6eb8-43cc-9165-358872497f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:52.541751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:52.604594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 574/1148] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:52.872586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fef5da-127e-4d9d-9d8c-4f88d7c35736 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:52.989245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fef5da-127e-4d9d-9d8c-4f88d7c35736 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:53.025320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0687db7-e976-4f6d-aecd-202da148679a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:53.026341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0687db7-e976-4f6d-aecd-202da148679a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:53.026758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0687db7-e976-4f6d-aecd-202da148679a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:53.027165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0687db7-e976-4f6d-aecd-202da148679a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:53.045159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0687db7-e976-4f6d-aecd-202da148679a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-e0687db7-e976-4f6d-aecd-202da148679a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:53.050820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:53.051924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 13:59:53.052766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 575/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:59:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:53.058917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3cc73-6eb8-43cc-9165-358872497f53 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:53.172774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb3cc73-6eb8-43cc-9165-358872497f53 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:53.207828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:53.208437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:53.208707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:53.208965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:53.209622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:53.209947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:53.211316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:53.220154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 575/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:59:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:53.229020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0687db7-e976-4f6d-aecd-202da148679a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:53.270297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:53.342476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0687db7-e976-4f6d-aecd-202da148679a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:53.379700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a2965-5f8a-40e5-826b-35f3f5c0de36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:53.380327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a2965-5f8a-40e5-826b-35f3f5c0de36 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:53.380602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a2965-5f8a-40e5-826b-35f3f5c0de36 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:53.380875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a2965-5f8a-40e5-826b-35f3f5c0de36 None admin] PATH_INFO: `/v3/users/d5b399661e6441ea813446f1372fa17e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:53.382070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a2965-5f8a-40e5-826b-35f3f5c0de36 None admin] RBAC: Authorizing `identity:get_user(user_id=d5b399661e6441ea813446f1372fa17e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:53.383513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a2965-5f8a-40e5-826b-35f3f5c0de36 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:53.385093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 576/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:53 2022] GET /identity/v3/users/d5b399661e6441ea813446f1372fa17e => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:53.392171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:53.410503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:53.501688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eb0a90-2aef-47ff-b2d0-a09b62e3337d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:53.537457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe2a531-10d4-4f44-8ad8-3b06d943a90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:53.538170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2a531-10d4-4f44-8ad8-3b06d943a90f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:53.538406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2a531-10d4-4f44-8ad8-3b06d943a90f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:53.538675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe2a531-10d4-4f44-8ad8-3b06d943a90f None admin] PATH_INFO: `/v3/projects/b2c92895df174b3485ed3c80bd929970` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:53.539839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2a531-10d4-4f44-8ad8-3b06d943a90f None admin] RBAC: Authorizing `identity:get_project(project_id=b2c92895df174b3485ed3c80bd929970)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:53.541297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe2a531-10d4-4f44-8ad8-3b06d943a90f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:53.542988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 576/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:53 2022] GET /identity/v3/projects/b2c92895df174b3485ed3c80bd929970 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:53.549438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a2965-5f8a-40e5-826b-35f3f5c0de36 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:53.652167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:53.655718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a2965-5f8a-40e5-826b-35f3f5c0de36 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:53.692760 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d891a83-b8d7-4d9e-ab84-c685baaa865e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:53.693403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d891a83-b8d7-4d9e-ab84-c685baaa865e None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:53.693679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d891a83-b8d7-4d9e-ab84-c685baaa865e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:53.693975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d891a83-b8d7-4d9e-ab84-c685baaa865e None admin] PATH_INFO: `/v3/projects/b2c92895df174b3485ed3c80bd929970/users/d5b399661e6441ea813446f1372fa17e/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:53.695966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d891a83-b8d7-4d9e-ab84-c685baaa865e None admin] RBAC: Authorizing `identity:create_grant(project_id=b2c92895df174b3485ed3c80bd929970, user_id=d5b399661e6441ea813446f1372fa17e, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:53.697401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d891a83-b8d7-4d9e-ab84-c685baaa865e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:53.713434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 577/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 13:59:53 2022] PUT /identity/v3/projects/b2c92895df174b3485ed3c80bd929970/users/d5b399661e6441ea813446f1372fa17e/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 13:59:55.797465 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31157-f709-49e1-94d6-e0fcb307dcbd None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:55.798081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31157-f709-49e1-94d6-e0fcb307dcbd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:55.798081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac31157-f709-49e1-94d6-e0fcb307dcbd None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:55.799297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 577/1154] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:55.805704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4808-f823-4433-bfaf-0c34ce51c253 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:55.806393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4808-f823-4433-bfaf-0c34ce51c253 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:55.806571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4808-f823-4433-bfaf-0c34ce51c253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:55.827479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-22ad4808-f823-4433-bfaf-0c34ce51c253 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:55.828678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-22ad4808-f823-4433-bfaf-0c34ce51c253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:55.897152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 578/1155] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:56.158426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31157-f709-49e1-94d6-e0fcb307dcbd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:56.254396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:56.278013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac31157-f709-49e1-94d6-e0fcb307dcbd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:56.316811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a05c-9aa5-4cd2-a822-8f4f072e950c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:56.317434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a05c-9aa5-4cd2-a822-8f4f072e950c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:56.317711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a05c-9aa5-4cd2-a822-8f4f072e950c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:56.317999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df5a05c-9aa5-4cd2-a822-8f4f072e950c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:56.329841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df5a05c-9aa5-4cd2-a822-8f4f072e950c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-5df5a05c-9aa5-4cd2-a822-8f4f072e950c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 13:59:56.334551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 13:59:56.335573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 13:59:56.336757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 578/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 13:59:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 13:59:56.345217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad4808-f823-4433-bfaf-0c34ce51c253 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:56.433101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:56.444091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad4808-f823-4433-bfaf-0c34ce51c253 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:56.478862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:56.479464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:56.479739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:56.480005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:56.480659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:56.480957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:56.482363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:56.490680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 579/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 13:59:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:56.496886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a05c-9aa5-4cd2-a822-8f4f072e950c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:56.601923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df5a05c-9aa5-4cd2-a822-8f4f072e950c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:56.639123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee2c43-475d-4d04-b050-5efe06898352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:56.639769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee2c43-475d-4d04-b050-5efe06898352 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:56.640037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee2c43-475d-4d04-b050-5efe06898352 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:56.640306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee2c43-475d-4d04-b050-5efe06898352 None admin] PATH_INFO: `/v3/users/d5b399661e6441ea813446f1372fa17e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:56.641600 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee2c43-475d-4d04-b050-5efe06898352 None admin] RBAC: Authorizing `identity:get_user(user_id=d5b399661e6441ea813446f1372fa17e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:56.643061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee2c43-475d-4d04-b050-5efe06898352 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:56.644685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 579/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 13:59:56 2022] GET /identity/v3/users/d5b399661e6441ea813446f1372fa17e => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:56.654259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:56.762807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee331e2-f587-411e-b475-47c49949602b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:56.804225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379aa75-5715-474a-99bb-f2cd26e5c7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:56.804832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379aa75-5715-474a-99bb-f2cd26e5c7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:56.805103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379aa75-5715-474a-99bb-f2cd26e5c7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:56.805367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379aa75-5715-474a-99bb-f2cd26e5c7e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:56.826952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:56.883906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379aa75-5715-474a-99bb-f2cd26e5c7e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:56.885352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379aa75-5715-474a-99bb-f2cd26e5c7e6 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:56.996119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 580/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 13:59:57.005292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee2c43-475d-4d04-b050-5efe06898352 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:57.018275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:57.116283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee2c43-475d-4d04-b050-5efe06898352 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:57.150998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdecc043-3267-47be-ab99-17aa1db63406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:57.151618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecc043-3267-47be-ab99-17aa1db63406 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:57.151888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecc043-3267-47be-ab99-17aa1db63406 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:57.152161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdecc043-3267-47be-ab99-17aa1db63406 None admin] PATH_INFO: `/v3/projects/b2c92895df174b3485ed3c80bd929970` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:57.153392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecc043-3267-47be-ab99-17aa1db63406 None admin] RBAC: Authorizing `identity:get_project(project_id=b2c92895df174b3485ed3c80bd929970)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:57.154840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdecc043-3267-47be-ab99-17aa1db63406 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:57.156495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 580/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 13:59:57 2022] GET /identity/v3/projects/b2c92895df174b3485ed3c80bd929970 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:57.162833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379aa75-5715-474a-99bb-f2cd26e5c7e6 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:57.216885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:57.266445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379aa75-5715-474a-99bb-f2cd26e5c7e6 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:57.308566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01753d91-ee4e-44ab-afa7-2fd6f6046876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:57.309070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01753d91-ee4e-44ab-afa7-2fd6f6046876 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:57.309339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01753d91-ee4e-44ab-afa7-2fd6f6046876 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:57.309605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01753d91-ee4e-44ab-afa7-2fd6f6046876 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:57.310348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01753d91-ee4e-44ab-afa7-2fd6f6046876 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=b2c92895df174b3485ed3c80bd929970, user.id=d5b399661e6441ea813446f1372fa17e) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 13:59:57.310647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01753d91-ee4e-44ab-afa7-2fd6f6046876 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 13:59:57.312078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01753d91-ee4e-44ab-afa7-2fd6f6046876 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 13:59:57.317973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 581/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 13:59:57 2022] GET /identity/v3/role_assignments?user.id=d5b399661e6441ea813446f1372fa17e&scope.project.id=b2c92895df174b3485ed3c80bd929970&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 13:59:59.481285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402fa2e-6f8d-4486-b7e0-2fa0e3af7631 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:59.481940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402fa2e-6f8d-4486-b7e0-2fa0e3af7631 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:59.481940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2402fa2e-6f8d-4486-b7e0-2fa0e3af7631 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:59.483171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 581/1162] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 13:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 13:59:59.489908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d07c5-cb99-45f1-9c7f-b299afebb3ad None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:59.490690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d07c5-cb99-45f1-9c7f-b299afebb3ad None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:59.490840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d07c5-cb99-45f1-9c7f-b299afebb3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 13:59:59.514592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-991d07c5-cb99-45f1-9c7f-b299afebb3ad None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 13:59:59.515835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-991d07c5-cb99-45f1-9c7f-b299afebb3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 13:59:59.584496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 582/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 13:59:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 13:59:59.848036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402fa2e-6f8d-4486-b7e0-2fa0e3af7631 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 13:59:59.931382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 13:59:59.958140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2402fa2e-6f8d-4486-b7e0-2fa0e3af7631 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 13:59:59.995486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52acc6-1ea9-4b2d-aaa4-17578d4e1027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 13:59:59.996079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52acc6-1ea9-4b2d-aaa4-17578d4e1027 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 13:59:59.996348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52acc6-1ea9-4b2d-aaa4-17578d4e1027 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 13:59:59.996611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52acc6-1ea9-4b2d-aaa4-17578d4e1027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:00.066547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52acc6-1ea9-4b2d-aaa4-17578d4e1027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:00.067977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a52acc6-1ea9-4b2d-aaa4-17578d4e1027 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:00.140197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:00.167669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 582/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 13:59:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 14:00:00.174107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d07c5-cb99-45f1-9c7f-b299afebb3ad None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:00.220803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:00.275386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d07c5-cb99-45f1-9c7f-b299afebb3ad None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:00.313591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf5ad56-963d-40ba-9ac3-38c930de1f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:00.314248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ad56-963d-40ba-9ac3-38c930de1f75 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:00.314521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ad56-963d-40ba-9ac3-38c930de1f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:00.314785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf5ad56-963d-40ba-9ac3-38c930de1f75 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:00.321576 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ad56-963d-40ba-9ac3-38c930de1f75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:00.323062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf5ad56-963d-40ba-9ac3-38c930de1f75 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-baf5ad56-963d-40ba-9ac3-38c930de1f75 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:00:00.334828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:00:00.336005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:00:00.336947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 583/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 14:00:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:00:00.343623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52acc6-1ea9-4b2d-aaa4-17578d4e1027 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:00.444872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52acc6-1ea9-4b2d-aaa4-17578d4e1027 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:00.480791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57626453-c96c-49a4-856c-9e0b953296c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:00.481382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626453-c96c-49a4-856c-9e0b953296c8 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:00.482021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626453-c96c-49a4-856c-9e0b953296c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:00.482021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57626453-c96c-49a4-856c-9e0b953296c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:00.482554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57626453-c96c-49a4-856c-9e0b953296c8 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:00:00.482856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57626453-c96c-49a4-856c-9e0b953296c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:00.484230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57626453-c96c-49a4-856c-9e0b953296c8 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:00.492271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 583/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Wed Oct 5 14:00:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:02.615110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17add1-9486-4104-8dd1-c8cb85a8f61f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:02.615716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17add1-9486-4104-8dd1-c8cb85a8f61f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:02.615716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17add1-9486-4104-8dd1-c8cb85a8f61f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:02.617000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 584/1167] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:00:02.623662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9c2a1-4209-4ee1-a5f0-fcdc58bca0b6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:02.624500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9c2a1-4209-4ee1-a5f0-fcdc58bca0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:02.624500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a9c2a1-4209-4ee1-a5f0-fcdc58bca0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:02.644128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-58a9c2a1-4209-4ee1-a5f0-fcdc58bca0b6 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:00:02.645314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-58a9c2a1-4209-4ee1-a5f0-fcdc58bca0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:00:02.676820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:02.718615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 584/1168] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:00:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:00:02.982229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17add1-9486-4104-8dd1-c8cb85a8f61f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:03.110074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17add1-9486-4104-8dd1-c8cb85a8f61f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:03.116865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:03.145008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f76d7-76a3-4244-8fa3-798ce06b3ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:03.145601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f76d7-76a3-4244-8fa3-798ce06b3ce3 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:03.145905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f76d7-76a3-4244-8fa3-798ce06b3ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:03.146164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f76d7-76a3-4244-8fa3-798ce06b3ce3 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:03.146822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f76d7-76a3-4244-8fa3-798ce06b3ce3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:03.148180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00f76d7-76a3-4244-8fa3-798ce06b3ce3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:03.169033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 585/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Wed Oct 5 14:00:02 2022] POST /identity/v3/domains => generated 267 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:00:05.291170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2764b4-90c4-4a13-8cac-d90fb2c404be None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:05.292075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2764b4-90c4-4a13-8cac-d90fb2c404be None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:05.292075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2764b4-90c4-4a13-8cac-d90fb2c404be None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:05.294234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 585/1170] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:00:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:00:05.302346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c4d1-3249-4539-91b1-40ee94e1e47c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:05.302800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c4d1-3249-4539-91b1-40ee94e1e47c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:05.302904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e101c4d1-3249-4539-91b1-40ee94e1e47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:05.322175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e101c4d1-3249-4539-91b1-40ee94e1e47c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:00:05.323347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e101c4d1-3249-4539-91b1-40ee94e1e47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:00:05.393271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 586/1171] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:00:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 14:00:05.656813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2764b4-90c4-4a13-8cac-d90fb2c404be None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:05.770392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2764b4-90c4-4a13-8cac-d90fb2c404be None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:05.805075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094f3c9f-faa7-485f-bfd0-75154263b1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:05.805775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f3c9f-faa7-485f-bfd0-75154263b1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:05.806051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f3c9f-faa7-485f-bfd0-75154263b1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:05.806310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094f3c9f-faa7-485f-bfd0-75154263b1c5 None admin] PATH_INFO: `/v3/domains/64ad00f52a2a459ba67a16e989ddd4a5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:05.817515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f3c9f-faa7-485f-bfd0-75154263b1c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=64ad00f52a2a459ba67a16e989ddd4a5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:05.819098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094f3c9f-faa7-485f-bfd0-75154263b1c5 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:05.820655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 586/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Wed Oct 5 14:00:05 2022] GET /identity/v3/domains/64ad00f52a2a459ba67a16e989ddd4a5 => generated 267 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:05.827384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101c4d1-3249-4539-91b1-40ee94e1e47c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:05.929101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:05.937561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e101c4d1-3249-4539-91b1-40ee94e1e47c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:05.971493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e96a4c-1c74-4522-bd90-e686d2cc37d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:05.972097 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a4c-1c74-4522-bd90-e686d2cc37d0 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:05.972369 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a4c-1c74-4522-bd90-e686d2cc37d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:05.972633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e96a4c-1c74-4522-bd90-e686d2cc37d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:05.973272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e96a4c-1c74-4522-bd90-e686d2cc37d0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:05.974746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e96a4c-1c74-4522-bd90-e686d2cc37d0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:06.009142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 587/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 14:00:05 2022] POST /identity/v3/projects => generated 361 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:00:08.149886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60472-a8ef-4d65-898d-b22ceac3f050 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:08.150500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60472-a8ef-4d65-898d-b22ceac3f050 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:08.150500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b60472-a8ef-4d65-898d-b22ceac3f050 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:08.151801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 587/1174] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:00:08.158335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0759b97-9e60-4732-aecf-008beaea297f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:08.159096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0759b97-9e60-4732-aecf-008beaea297f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:08.159241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0759b97-9e60-4732-aecf-008beaea297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:08.186168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d0759b97-9e60-4732-aecf-008beaea297f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:00:08.187371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d0759b97-9e60-4732-aecf-008beaea297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:00:08.263659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 588/1175] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:00:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:00:08.525110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b60472-a8ef-4d65-898d-b22ceac3f050 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:08.564566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:08.648202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b60472-a8ef-4d65-898d-b22ceac3f050 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:08.686014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e3e613-5a29-4897-a469-d1f650f31fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:08.686631 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3e613-5a29-4897-a469-d1f650f31fc4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:08.686899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3e613-5a29-4897-a469-d1f650f31fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:08.687246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e3e613-5a29-4897-a469-d1f650f31fc4 None admin] PATH_INFO: `/v3/domains/64ad00f52a2a459ba67a16e989ddd4a5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:08.688393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3e613-5a29-4897-a469-d1f650f31fc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=64ad00f52a2a459ba67a16e989ddd4a5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:08.689851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e3e613-5a29-4897-a469-d1f650f31fc4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:08.691782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 588/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Wed Oct 5 14:00:08 2022] GET /identity/v3/domains/64ad00f52a2a459ba67a16e989ddd4a5 => generated 267 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:08.699947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0759b97-9e60-4732-aecf-008beaea297f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:08.788244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:08.812300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0759b97-9e60-4732-aecf-008beaea297f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:08.850937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838079f-34d1-40cd-b95b-513a321de820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:08.851557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838079f-34d1-40cd-b95b-513a321de820 None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:08.851835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838079f-34d1-40cd-b95b-513a321de820 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:08.852100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838079f-34d1-40cd-b95b-513a321de820 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:08.852751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838079f-34d1-40cd-b95b-513a321de820 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:08.854239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838079f-34d1-40cd-b95b-513a321de820 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:08.882874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 589/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 14:00:08 2022] POST /identity/v3/users => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:00:11.099248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456f5f4-a2d6-4a7e-b7cf-4e3558a6c990 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:11.099816 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456f5f4-a2d6-4a7e-b7cf-4e3558a6c990 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:11.099816 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456f5f4-a2d6-4a7e-b7cf-4e3558a6c990 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:11.101129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 589/1178] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:00:11.107527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12630-b2e2-45cb-a964-f3155e852efe None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:11.108237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12630-b2e2-45cb-a964-f3155e852efe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:11.108389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e12630-b2e2-45cb-a964-f3155e852efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:11.139655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-08e12630-b2e2-45cb-a964-f3155e852efe None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:00:11.141106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-08e12630-b2e2-45cb-a964-f3155e852efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:00:11.214037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 590/1179] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:00:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:00:11.485229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456f5f4-a2d6-4a7e-b7cf-4e3558a6c990 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:11.596527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:11.609055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456f5f4-a2d6-4a7e-b7cf-4e3558a6c990 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:11.650885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc4a4d-dfda-4bec-b04f-5fa4eaec231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:11.651505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc4a4d-dfda-4bec-b04f-5fa4eaec231b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:11.651776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc4a4d-dfda-4bec-b04f-5fa4eaec231b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:11.652039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc4a4d-dfda-4bec-b04f-5fa4eaec231b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:11.663311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc4a4d-dfda-4bec-b04f-5fa4eaec231b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-ebcc4a4d-dfda-4bec-b04f-5fa4eaec231b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:00:11.667925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:00:11.668966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:00:11.669867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 590/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 14:00:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:00:11.675720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12630-b2e2-45cb-a964-f3155e852efe None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:11.743357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:11.784794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e12630-b2e2-45cb-a964-f3155e852efe None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:11.823565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:11.824208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:11.824485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:11.824761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:11.825415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:00:11.825737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:11.827148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:11.835438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 591/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 14:00:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:11.841801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc4a4d-dfda-4bec-b04f-5fa4eaec231b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:11.954380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc4a4d-dfda-4bec-b04f-5fa4eaec231b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:12.000218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d5908-ccc8-4f4b-9e16-fab21a7281dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:12.000768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d5908-ccc8-4f4b-9e16-fab21a7281dd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:12.001045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d5908-ccc8-4f4b-9e16-fab21a7281dd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:12.001314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d5908-ccc8-4f4b-9e16-fab21a7281dd None admin] PATH_INFO: `/v3/users/622e638e4b6c43798f822b0a01d6bd24` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:12.014869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d5908-ccc8-4f4b-9e16-fab21a7281dd None admin] RBAC: Authorizing `identity:get_user(user_id=622e638e4b6c43798f822b0a01d6bd24)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:12.016412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d5908-ccc8-4f4b-9e16-fab21a7281dd None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:12.018018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 591/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 14:00:11 2022] GET /identity/v3/users/622e638e4b6c43798f822b0a01d6bd24 => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:12.026714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:12.138025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5237c5-b32b-4a2b-aa05-b4d59c355d72 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:12.164872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:12.174551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c485d0-8881-47e1-9141-7512ad97dacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:12.175155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c485d0-8881-47e1-9141-7512ad97dacd None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:12.175431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c485d0-8881-47e1-9141-7512ad97dacd None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:12.175697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c485d0-8881-47e1-9141-7512ad97dacd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:12.244554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c485d0-8881-47e1-9141-7512ad97dacd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:12.246052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c485d0-8881-47e1-9141-7512ad97dacd None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:12.366630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 592/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 14:00:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 14:00:12.372956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d5908-ccc8-4f4b-9e16-fab21a7281dd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:12.395688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:12.506367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d5908-ccc8-4f4b-9e16-fab21a7281dd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:12.547749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d344b53-9141-4853-b99c-b381b94b0aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:12.548369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d344b53-9141-4853-b99c-b381b94b0aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:12.548645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d344b53-9141-4853-b99c-b381b94b0aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:12.548915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d344b53-9141-4853-b99c-b381b94b0aa1 None admin] PATH_INFO: `/v3/projects/0c09732134a64cbcb72f653a182293cb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:12.550117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d344b53-9141-4853-b99c-b381b94b0aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=0c09732134a64cbcb72f653a182293cb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:12.551564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d344b53-9141-4853-b99c-b381b94b0aa1 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:12.553244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 592/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 14:00:12 2022] GET /identity/v3/projects/0c09732134a64cbcb72f653a182293cb => generated 361 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:12.563610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c485d0-8881-47e1-9141-7512ad97dacd None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:12.599987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:12.678071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c485d0-8881-47e1-9141-7512ad97dacd None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:12.715960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4f08a-8e46-4791-9bd4-5e6b4d3644af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:12.716551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4f08a-8e46-4791-9bd4-5e6b4d3644af None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:12.716824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4f08a-8e46-4791-9bd4-5e6b4d3644af None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:12.717088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4f08a-8e46-4791-9bd4-5e6b4d3644af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:12.717950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4f08a-8e46-4791-9bd4-5e6b4d3644af None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=0c09732134a64cbcb72f653a182293cb, user.id=622e638e4b6c43798f822b0a01d6bd24) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:00:12.718275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4f08a-8e46-4791-9bd4-5e6b4d3644af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:12.719811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4f08a-8e46-4791-9bd4-5e6b4d3644af None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:12.725273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 593/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 14:00:12 2022] GET /identity/v3/role_assignments?user.id=622e638e4b6c43798f822b0a01d6bd24&scope.project.id=0c09732134a64cbcb72f653a182293cb&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:14.882824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcd24d-5698-4e4c-8a57-3bca10f5a4b2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:14.883457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcd24d-5698-4e4c-8a57-3bca10f5a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:14.883457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fcd24d-5698-4e4c-8a57-3bca10f5a4b2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:14.884612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 593/1186] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:00:14.891297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19ebec-0b21-4ba4-8740-e8ad6eec7992 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:14.892014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19ebec-0b21-4ba4-8740-e8ad6eec7992 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:14.892173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a19ebec-0b21-4ba4-8740-e8ad6eec7992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:14.921558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3a19ebec-0b21-4ba4-8740-e8ad6eec7992 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:00:14.922775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3a19ebec-0b21-4ba4-8740-e8ad6eec7992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:00:14.994779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 594/1187] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:00:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:00:15.275147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcd24d-5698-4e4c-8a57-3bca10f5a4b2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:15.364139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:15.388720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fcd24d-5698-4e4c-8a57-3bca10f5a4b2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:15.423065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00d402-f76a-4def-aed3-dbcfd75c8156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:15.423687 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00d402-f76a-4def-aed3-dbcfd75c8156 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:15.423964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00d402-f76a-4def-aed3-dbcfd75c8156 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:15.424234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00d402-f76a-4def-aed3-dbcfd75c8156 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:15.435209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00d402-f76a-4def-aed3-dbcfd75c8156 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-bd00d402-f76a-4def-aed3-dbcfd75c8156 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:00:15.440006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:00:15.441083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:00:15.442314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 594/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 14:00:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:00:15.449927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19ebec-0b21-4ba4-8740-e8ad6eec7992 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:15.472675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:15.552259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a19ebec-0b21-4ba4-8740-e8ad6eec7992 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:15.597518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:15.598159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:15.598440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:15.598705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:15.599361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:00:15.599650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:15.601015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:15.609065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 595/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 14:00:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:15.617072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00d402-f76a-4def-aed3-dbcfd75c8156 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:15.727104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00d402-f76a-4def-aed3-dbcfd75c8156 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:15.761737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94feb2-e5f5-4e07-b0fb-d83acf19c6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:15.762351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94feb2-e5f5-4e07-b0fb-d83acf19c6ce None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:15.762626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94feb2-e5f5-4e07-b0fb-d83acf19c6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:15.762893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94feb2-e5f5-4e07-b0fb-d83acf19c6ce None admin] PATH_INFO: `/v3/users/622e638e4b6c43798f822b0a01d6bd24` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:15.764081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94feb2-e5f5-4e07-b0fb-d83acf19c6ce None admin] RBAC: Authorizing `identity:get_user(user_id=622e638e4b6c43798f822b0a01d6bd24)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:15.765569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba94feb2-e5f5-4e07-b0fb-d83acf19c6ce None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:15.767150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 595/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 14:00:15 2022] GET /identity/v3/users/622e638e4b6c43798f822b0a01d6bd24 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:15.775904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:15.864347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:15.888320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03cb643-32ab-4a4e-9e8c-a36b040bc832 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:15.923636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eb47de-ab9f-4a27-9530-ea7d6dd9f774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:15.924218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb47de-ab9f-4a27-9530-ea7d6dd9f774 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:15.924509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb47de-ab9f-4a27-9530-ea7d6dd9f774 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:15.924775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eb47de-ab9f-4a27-9530-ea7d6dd9f774 None admin] PATH_INFO: `/v3/projects/0c09732134a64cbcb72f653a182293cb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:15.925949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb47de-ab9f-4a27-9530-ea7d6dd9f774 None admin] RBAC: Authorizing `identity:get_project(project_id=0c09732134a64cbcb72f653a182293cb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:15.927391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eb47de-ab9f-4a27-9530-ea7d6dd9f774 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:15.928981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 596/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 14:00:15 2022] GET /identity/v3/projects/0c09732134a64cbcb72f653a182293cb => generated 361 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:15.938750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94feb2-e5f5-4e07-b0fb-d83acf19c6ce None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:15.952126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:16.042019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94feb2-e5f5-4e07-b0fb-d83acf19c6ce None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:16.076811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d3292-d215-4084-ac5b-47b774e1dde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:16.077467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3292-d215-4084-ac5b-47b774e1dde6 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:16.077760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3292-d215-4084-ac5b-47b774e1dde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:16.078033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d3292-d215-4084-ac5b-47b774e1dde6 None admin] PATH_INFO: `/v3/projects/0c09732134a64cbcb72f653a182293cb/users/622e638e4b6c43798f822b0a01d6bd24/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:16.080166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d3292-d215-4084-ac5b-47b774e1dde6 None admin] RBAC: Authorizing `identity:create_grant(project_id=0c09732134a64cbcb72f653a182293cb, user_id=622e638e4b6c43798f822b0a01d6bd24, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:16.081580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d3292-d215-4084-ac5b-47b774e1dde6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:16.100844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 596/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 14:00:15 2022] PUT /identity/v3/projects/0c09732134a64cbcb72f653a182293cb/users/622e638e4b6c43798f822b0a01d6bd24/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:00:18.212540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d00214-57d5-4bdc-987b-708763830279 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:18.213149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d00214-57d5-4bdc-987b-708763830279 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:18.213149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d00214-57d5-4bdc-987b-708763830279 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:18.214474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 597/1193] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:00:18.221038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea294cf-7c1a-4906-9943-b0227ccf2e99 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:18.221812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea294cf-7c1a-4906-9943-b0227ccf2e99 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:18.221971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea294cf-7c1a-4906-9943-b0227ccf2e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:18.252372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9ea294cf-7c1a-4906-9943-b0227ccf2e99 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:00:18.253569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9ea294cf-7c1a-4906-9943-b0227ccf2e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:00:18.318387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:18.323608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 597/1194] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:00:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:00:18.595421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d00214-57d5-4bdc-987b-708763830279 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:18.713662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d00214-57d5-4bdc-987b-708763830279 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:18.749402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e55584-8680-4d42-9217-2aacf2c8f02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:18.750052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e55584-8680-4d42-9217-2aacf2c8f02a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:18.750319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e55584-8680-4d42-9217-2aacf2c8f02a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:18.750583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e55584-8680-4d42-9217-2aacf2c8f02a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:18.762223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e55584-8680-4d42-9217-2aacf2c8f02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-b7e55584-8680-4d42-9217-2aacf2c8f02a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:00:18.766988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:00:18.768042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:00:18.768903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 598/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 14:00:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:00:18.775066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea294cf-7c1a-4906-9943-b0227ccf2e99 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:18.887099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea294cf-7c1a-4906-9943-b0227ccf2e99 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:18.919968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:18.920567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:18.920839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:18.921106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:18.921937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:00:18.922244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:18.923614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:18.931847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 598/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 14:00:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:18.939909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e55584-8680-4d42-9217-2aacf2c8f02a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:18.954137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:19.052815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e55584-8680-4d42-9217-2aacf2c8f02a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:19.090213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fc98d-f043-4979-abb1-3d2819d79e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:19.090838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc98d-f043-4979-abb1-3d2819d79e09 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:19.091113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc98d-f043-4979-abb1-3d2819d79e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:19.091380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007fc98d-f043-4979-abb1-3d2819d79e09 None admin] PATH_INFO: `/v3/users/622e638e4b6c43798f822b0a01d6bd24` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:19.092568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fc98d-f043-4979-abb1-3d2819d79e09 None admin] RBAC: Authorizing `identity:get_user(user_id=622e638e4b6c43798f822b0a01d6bd24)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:19.094250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007fc98d-f043-4979-abb1-3d2819d79e09 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:19.095803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 599/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Wed Oct 5 14:00:18 2022] GET /identity/v3/users/622e638e4b6c43798f822b0a01d6bd24 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:19.104443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:19.181293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:19.223260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca8823-21b8-4f21-be9e-4d5c3051e6a2 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:19.258070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd29d2-be41-4855-93e7-e8384a4c3baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:19.258663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd29d2-be41-4855-93e7-e8384a4c3baf None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:19.258934 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd29d2-be41-4855-93e7-e8384a4c3baf None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:19.259198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fd29d2-be41-4855-93e7-e8384a4c3baf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:19.333834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd29d2-be41-4855-93e7-e8384a4c3baf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:19.335227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fd29d2-be41-4855-93e7-e8384a4c3baf None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:19.397460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:19.440498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 599/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 14:00:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 14:00:19.446743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fc98d-f043-4979-abb1-3d2819d79e09 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:19.531687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:19.556062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007fc98d-f043-4979-abb1-3d2819d79e09 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:19.590015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c83dbf1-f857-4729-82f6-2429b2848736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:19.590579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83dbf1-f857-4729-82f6-2429b2848736 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:19.590852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83dbf1-f857-4729-82f6-2429b2848736 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:19.591121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c83dbf1-f857-4729-82f6-2429b2848736 None admin] PATH_INFO: `/v3/projects/0c09732134a64cbcb72f653a182293cb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:19.592280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83dbf1-f857-4729-82f6-2429b2848736 None admin] RBAC: Authorizing `identity:get_project(project_id=0c09732134a64cbcb72f653a182293cb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:19.593800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c83dbf1-f857-4729-82f6-2429b2848736 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:19.595398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 600/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Wed Oct 5 14:00:19 2022] GET /identity/v3/projects/0c09732134a64cbcb72f653a182293cb => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:19.602080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd29d2-be41-4855-93e7-e8384a4c3baf None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:00:19.712833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fd29d2-be41-4855-93e7-e8384a4c3baf None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:00:19.748971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9948d-2e03-4fd5-8dee-172a05603429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:00:19.749626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9948d-2e03-4fd5-8dee-172a05603429 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:00:19.749933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9948d-2e03-4fd5-8dee-172a05603429 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:00:19.750205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9948d-2e03-4fd5-8dee-172a05603429 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:00:19.750925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9948d-2e03-4fd5-8dee-172a05603429 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=0c09732134a64cbcb72f653a182293cb, user.id=622e638e4b6c43798f822b0a01d6bd24) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:00:19.751224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9948d-2e03-4fd5-8dee-172a05603429 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:00:19.752652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9948d-2e03-4fd5-8dee-172a05603429 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:00:19.758469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 600/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 14:00:19 2022] GET /identity/v3/role_assignments?user.id=622e638e4b6c43798f822b0a01d6bd24&scope.project.id=0c09732134a64cbcb72f653a182293cb&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:00:29.585204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=27 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:00:29.755831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:15.496684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213ae8-228f-401a-8ffa-41b7b62136f7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:15.497317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213ae8-228f-401a-8ffa-41b7b62136f7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:15.497317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26213ae8-228f-401a-8ffa-41b7b62136f7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:15.498640 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 601/1201] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:15.509266 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339b33-4ac1-4fc8-a343-21779324ab1a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:15.509941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339b33-4ac1-4fc8-a343-21779324ab1a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:15.509941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d339b33-4ac1-4fc8-a343-21779324ab1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:15.530265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0d339b33-4ac1-4fc8-a343-21779324ab1a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:15.531572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0d339b33-4ac1-4fc8-a343-21779324ab1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:15.617694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 601/1202] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:15.880499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26213ae8-228f-401a-8ffa-41b7b62136f7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:16.002211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26213ae8-228f-401a-8ffa-41b7b62136f7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:16.058660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:16.059494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:16.059781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:16.060151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:16.072711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:16.074494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:16.091618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:16.092816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:16.094833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:16.094833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:16.094833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:16.094833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 602/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:16.099833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d339b33-4ac1-4fc8-a343-21779324ab1a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:16.239192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d339b33-4ac1-4fc8-a343-21779324ab1a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:16.270420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:16.277746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778c4d5-1269-4c06-b9f7-1760a1ee4277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:16.278420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c4d5-1269-4c06-b9f7-1760a1ee4277 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:16.278694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c4d5-1269-4c06-b9f7-1760a1ee4277 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:16.279032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778c4d5-1269-4c06-b9f7-1760a1ee4277 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:16.279680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778c4d5-1269-4c06-b9f7-1760a1ee4277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:16.279993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778c4d5-1269-4c06-b9f7-1760a1ee4277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:16.281461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778c4d5-1269-4c06-b9f7-1760a1ee4277 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:16.298885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 602/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:16.299867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:16.363886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:16.423032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc182e-2fb3-4a9e-ae6b-4c47be4fb009 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:16.465010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fa54c7-dd12-43da-8438-4fa67d740d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:16.465670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa54c7-dd12-43da-8438-4fa67d740d0b None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:16.466004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa54c7-dd12-43da-8438-4fa67d740d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:16.466278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fa54c7-dd12-43da-8438-4fa67d740d0b None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:16.467067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa54c7-dd12-43da-8438-4fa67d740d0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:16.468770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fa54c7-dd12-43da-8438-4fa67d740d0b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:16.493437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 603/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Wed Oct 5 14:01:16 2022] POST /identity/v3/users => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:01:19.052814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37148d40-ce97-4dd3-8f85-96845f96ba1a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:19.054672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37148d40-ce97-4dd3-8f85-96845f96ba1a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:19.054672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37148d40-ce97-4dd3-8f85-96845f96ba1a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:19.054998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 603/1206] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:19.061210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f1531-aeb3-479c-9868-e4766d169a6d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:19.061858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f1531-aeb3-479c-9868-e4766d169a6d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:19.061858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f1531-aeb3-479c-9868-e4766d169a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:19.083463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c38f1531-aeb3-479c-9868-e4766d169a6d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:19.085307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c38f1531-aeb3-479c-9868-e4766d169a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:19.163344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 604/1207] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:19.456131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37148d40-ce97-4dd3-8f85-96845f96ba1a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:19.573978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37148d40-ce97-4dd3-8f85-96845f96ba1a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:19.614702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5523f6-b041-4399-900a-b59200ab097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:19.615377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5523f6-b041-4399-900a-b59200ab097f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:19.615654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5523f6-b041-4399-900a-b59200ab097f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:19.615914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5523f6-b041-4399-900a-b59200ab097f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:19.628740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5523f6-b041-4399-900a-b59200ab097f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6c5523f6-b041-4399-900a-b59200ab097f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:01:19.633830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:19.634806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:19.635652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 604/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 14:01:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:19.641246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f1531-aeb3-479c-9868-e4766d169a6d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:19.683841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:19.753202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f1531-aeb3-479c-9868-e4766d169a6d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:19.788373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:19.789022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:19.789291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:19.789557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:19.790340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:19.790546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:19.792130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:19.801349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 605/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 14:01:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:19.807699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5523f6-b041-4399-900a-b59200ab097f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:19.865606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:19.923174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5523f6-b041-4399-900a-b59200ab097f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:19.961139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:19.961798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:19.962068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:19.962330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:19.972936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:19.974405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:19.990062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:19.991385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:19.992440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:19.992440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:19.992440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:19.992440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 605/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:19.997665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:20.147090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:20.150074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a237cd-5d72-49e1-8473-d2cb35d3288c None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:20.192956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:20.193676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:20.193952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:20.194235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:20.194900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:20.195237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:20.197202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:20.211340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 606/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:20.217249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:20.324412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b88d00-82b7-4fc2-b4db-bb2cac6eb27d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:20.326091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:20.359460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:20.360110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:20.360371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:20.360645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:20.368349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:20.369808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 14:01:20.381073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:20.382255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:20.383196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 606/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:20.388091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:20.503824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f91a7-cedb-4222-8c16-f899dd2c7f24 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:20.543610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:20.544235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:20.544504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:20.544782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:20.545480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:20.545827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:20.547250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:20.571116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 607/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 14:01:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:20.571340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:20.677476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b247c1-8c67-41eb-a9fb-d17fe9bd9647 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:20.721602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c676c31-8894-4b00-85c0-4f62f0dce194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:20.722263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c676c31-8894-4b00-85c0-4f62f0dce194 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:20.722538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c676c31-8894-4b00-85c0-4f62f0dce194 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:20.722853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c676c31-8894-4b00-85c0-4f62f0dce194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:20.757876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:20.802132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c676c31-8894-4b00-85c0-4f62f0dce194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:20.803784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c676c31-8894-4b00-85c0-4f62f0dce194 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:20.903964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 607/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 14:01:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 14:01:20.910232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:20.929403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:21.015902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec31c9-d8a9-440e-aaa2-085da1542dd5 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:21.054156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:21.055112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:21.055367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:21.055635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:21.066990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:21.068562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:21.083791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:21.084995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:21.086126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:21.086126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:21.086126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:21.086786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 608/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:21.092991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c676c31-8894-4b00-85c0-4f62f0dce194 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:21.160375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:21.206034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c676c31-8894-4b00-85c0-4f62f0dce194 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:21.244894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:21.245618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:21.245976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:21.246249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:21.246949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:21.247248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:21.248661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:21.265776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 608/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:21.267547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:21.343471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:21.377896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176b3e4d-d3ca-42f6-9adf-4a1c724f9d3b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:21.414759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:21.415412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:21.415683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:21.415972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:21.423347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:21.424812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 14:01:21.437527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:21.438863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:21.440047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 609/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 14:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:21.444849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:21.558378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fed8b4-b297-4586-89bb-a9d9593f8d48 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:21.582571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:21.594662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ac60-d636-438b-a532-4ad2e1304e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:21.595305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ac60-d636-438b-a532-4ad2e1304e09 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:21.595572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ac60-d636-438b-a532-4ad2e1304e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:21.595831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ac60-d636-438b-a532-4ad2e1304e09 None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:21.596510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5ac60-d636-438b-a532-4ad2e1304e09 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:21.596852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5ac60-d636-438b-a532-4ad2e1304e09 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:21.598310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e5ac60-d636-438b-a532-4ad2e1304e09 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:21.612226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 609/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 14:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:21.619128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:21.738676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a7267f-17cd-473f-822f-dd2308bd1e97 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:21.769828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:21.776263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec9db7b-a915-4434-a07b-d59931b2422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:21.776919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9db7b-a915-4434-a07b-d59931b2422e None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:21.777200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9db7b-a915-4434-a07b-d59931b2422e None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:21.777465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec9db7b-a915-4434-a07b-d59931b2422e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:21.778318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9db7b-a915-4434-a07b-d59931b2422e None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=5b9792d95cad4f0d8708df0a608f30ed) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:21.778618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9db7b-a915-4434-a07b-d59931b2422e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:21.780183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec9db7b-a915-4434-a07b-d59931b2422e None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:21.786052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 610/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 14:01:21 2022] GET /identity/v3/role_assignments?user.id=5b9792d95cad4f0d8708df0a608f30ed&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:23.956416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05308266-8a0a-444b-876c-d8f5c98a9745 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:23.956988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05308266-8a0a-444b-876c-d8f5c98a9745 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:23.956988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05308266-8a0a-444b-876c-d8f5c98a9745 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:23.958317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 610/1220] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:23.964113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c094a09-5e7c-45d6-889c-7f2436aad6a0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:23.964560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c094a09-5e7c-45d6-889c-7f2436aad6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:23.964685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c094a09-5e7c-45d6-889c-7f2436aad6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:23.984216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3c094a09-5e7c-45d6-889c-7f2436aad6a0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:23.985409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c094a09-5e7c-45d6-889c-7f2436aad6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:24.059760 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 611/1221] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:24.330384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05308266-8a0a-444b-876c-d8f5c98a9745 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:24.440277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05308266-8a0a-444b-876c-d8f5c98a9745 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:24.474825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075c4e-b759-4b32-aca5-ee6b7e7033dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:24.475455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075c4e-b759-4b32-aca5-ee6b7e7033dd None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:24.475729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075c4e-b759-4b32-aca5-ee6b7e7033dd None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:24.475996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06075c4e-b759-4b32-aca5-ee6b7e7033dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:24.486901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06075c4e-b759-4b32-aca5-ee6b7e7033dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-06075c4e-b759-4b32-aca5-ee6b7e7033dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:01:24.491888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:24.492841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:24.493885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 611/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 14:01:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:24.499935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c094a09-5e7c-45d6-889c-7f2436aad6a0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:24.602034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c094a09-5e7c-45d6-889c-7f2436aad6a0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:24.625946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:24.641193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:24.641828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:24.642100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:24.642357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:24.643030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] RBAC: Adding query filter params (name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:24.643346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:24.644716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:24.653061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 612/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 14:01:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:24.659437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075c4e-b759-4b32-aca5-ee6b7e7033dd None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:24.709276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:24.772628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06075c4e-b759-4b32-aca5-ee6b7e7033dd None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:24.806534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:24.807182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:24.807456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:24.807719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:24.818097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:24.819522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:24.833723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:24.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:24.835908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:24.835908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:24.835908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:24.839678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 612/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:24.841188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:24.943919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510719d3-6f82-4f84-9f1f-ab7a61c56380 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:24.977931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:24.978617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:24.978790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:24.979067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:24.979748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:24.979981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:24.981351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:24.993658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 613/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:24.999741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:25.101528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:25.109070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7210b80a-ba56-4d6a-9279-aac05b06db2b None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:25.146283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:25.146918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:25.147190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:25.147452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:25.155378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:25.156815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 14:01:25.168453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:25.169414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:25.170421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 613/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:25.175928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:25.225180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:25.281256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7122a55-f4d1-4587-9019-39d35a47c879 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:25.316854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:25.317461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:25.317784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:25.318049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:25.318767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:25.319063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:25.320481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:25.338141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 614/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 14:01:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:25.344345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:25.450466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bef225a-ec02-4280-9521-0fd9880cff33 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:25.486833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:25.487557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:25.487846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:25.488138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:25.504450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:25.505965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:25.521117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:25.522385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:25.523493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:25.523493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:25.523493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:25.523493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 614/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:25.528255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:25.635149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f1bb9-5428-4227-8627-9082c0ea7bf8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:25.637590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:25.674078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:25.674689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:25.674964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:25.675228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:25.675872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:25.676174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:25.677542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:25.689024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 615/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:25.695252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:25.704836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:25.797604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c388688c-6cb5-4c39-8025-b1b5a437f509 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:25.834764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:25.835436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:25.835742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:25.836079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:25.843544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:25.844979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 14:01:25.857177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:25.858622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:25.859665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 615/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 14:01:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:25.864575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:25.970130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b105cced-0c7f-4b5d-be34-f7bd9891eaf2 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:26.008151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74159621-aec4-45af-b0f6-331417c5f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:26.009593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159621-aec4-45af-b0f6-331417c5f963 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:26.009908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159621-aec4-45af-b0f6-331417c5f963 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:26.010252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74159621-aec4-45af-b0f6-331417c5f963 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:26.010965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74159621-aec4-45af-b0f6-331417c5f963 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:26.011237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74159621-aec4-45af-b0f6-331417c5f963 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:26.012831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74159621-aec4-45af-b0f6-331417c5f963 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:26.025912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 616/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 14:01:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:26.032886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:26.102057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:26.140299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811bcf1-8978-48e0-b013-23d471a2eb85 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:26.175237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3e604-72d8-4f2f-81d3-91f3a36c7de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:26.175868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e604-72d8-4f2f-81d3-91f3a36c7de6 None admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:26.176141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e604-72d8-4f2f-81d3-91f3a36c7de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:26.176410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3e604-72d8-4f2f-81d3-91f3a36c7de6 None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/5b9792d95cad4f0d8708df0a608f30ed/roles/7d39e08123cb40c78586e4d7eb3ed9ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:26.193299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3e604-72d8-4f2f-81d3-91f3a36c7de6 None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=5b9792d95cad4f0d8708df0a608f30ed, role_id=7d39e08123cb40c78586e4d7eb3ed9ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:26.195247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3e604-72d8-4f2f-81d3-91f3a36c7de6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:26.209661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 616/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 14:01:26 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/5b9792d95cad4f0d8708df0a608f30ed/roles/7d39e08123cb40c78586e4d7eb3ed9ba => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:01:28.417267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a97cc-462f-40a4-af11-e1cb4400ba8c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:28.417897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a97cc-462f-40a4-af11-e1cb4400ba8c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:28.417897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67a97cc-462f-40a4-af11-e1cb4400ba8c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:28.419161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 617/1233] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:28.424482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4e5bb-86fb-4713-9e51-2d5cf3f5473f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:28.424876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4e5bb-86fb-4713-9e51-2d5cf3f5473f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:28.425044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c4e5bb-86fb-4713-9e51-2d5cf3f5473f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:28.444143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a8c4e5bb-86fb-4713-9e51-2d5cf3f5473f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:28.445340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a8c4e5bb-86fb-4713-9e51-2d5cf3f5473f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:28.518074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 617/1234] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:28.781862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a97cc-462f-40a4-af11-e1cb4400ba8c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:28.807621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:28.886181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67a97cc-462f-40a4-af11-e1cb4400ba8c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:28.920108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8b40a-462a-42d2-84b4-9ad01a0fe99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:28.920787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8b40a-462a-42d2-84b4-9ad01a0fe99b None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:28.921073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8b40a-462a-42d2-84b4-9ad01a0fe99b None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:28.921321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8b40a-462a-42d2-84b4-9ad01a0fe99b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:28.931343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f8b40a-462a-42d2-84b4-9ad01a0fe99b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-d0f8b40a-462a-42d2-84b4-9ad01a0fe99b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:01:28.936112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:28.937151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:28.938319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 618/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Wed Oct 5 14:01:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:28.943331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4e5bb-86fb-4713-9e51-2d5cf3f5473f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:28.989232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:29.044144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c4e5bb-86fb-4713-9e51-2d5cf3f5473f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:29.081147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:29.081794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:29.082100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:29.082399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:29.083028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] RBAC: Adding query filter params (name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:29.083350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:29.084701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:29.095100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 618/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Wed Oct 5 14:01:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:29.101191 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8b40a-462a-42d2-84b4-9ad01a0fe99b None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:29.197607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:29.216317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8b40a-462a-42d2-84b4-9ad01a0fe99b None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:29.256404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:29.257035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:29.257312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:29.257575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:29.268532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:29.270120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:29.284780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:29.285942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:29.287171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:29.287171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:29.287171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:29.287171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 619/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:29.292126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:29.397574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccce49e9-ed3a-4dd3-9ec7-80acd452225c None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:29.407335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:29.434647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:29.435290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:29.435544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:29.435806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:29.436538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:29.436829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:29.438225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:29.452549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 619/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:29.458674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:29.568204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f934bef-7b1a-4589-8ea1-d483c14e9875 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:29.603166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:29.603858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:29.604167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:29.604460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:29.607746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:29.613309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:29.614965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 14:01:29.626296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:29.627462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:29.628768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 620/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:29.633290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:29.736739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51094c1-2264-438e-95e0-db9ef695bdad None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:29.774956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:29.775644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:29.775911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:29.776257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:29.776986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:29.777281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:29.779010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:29.798176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 620/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 14:01:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:29.804208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:29.913521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06168d87-51da-445d-91ed-24dcb28ea6f4 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:29.949478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d6e3f-207f-4817-9507-66fdc78c3e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:29.950114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d6e3f-207f-4817-9507-66fdc78c3e03 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:29.950388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d6e3f-207f-4817-9507-66fdc78c3e03 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:29.950653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d6e3f-207f-4817-9507-66fdc78c3e03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:30.015848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:30.024193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d6e3f-207f-4817-9507-66fdc78c3e03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:30.025732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1d6e3f-207f-4817-9507-66fdc78c3e03 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:30.127186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 621/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 14:01:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 14:01:30.134102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:30.165374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:30.247764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3a33d3-4509-455e-bc8a-76f02025e603 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:30.281616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:30.282205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:30.282477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:30.282741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:30.292643 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:30.294136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:30.308469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:30.309733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:30.310960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:30.310960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:30.310960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:30.310960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 621/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:30.315596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d6e3f-207f-4817-9507-66fdc78c3e03 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:30.408781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:30.415944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1d6e3f-207f-4817-9507-66fdc78c3e03 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:30.451881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:30.452586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:30.452884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:30.453156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:30.453820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:30.454118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:30.455492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:30.470068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 622/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:30.475915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:30.559007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:30.583421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586c852-99ae-4769-9ba9-7caa32cff34e None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:30.620830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:30.621527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:30.621871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:30.623022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:30.629256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:30.630760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 14:01:30.644688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:30.646239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:30.647340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 622/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 14:01:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:30.651867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:30.759552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e762d-20b6-4846-a661-796245a774bc None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:30.797179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19c6382-aa21-46d7-897e-5da399001fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:30.797809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6382-aa21-46d7-897e-5da399001fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:30.798084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6382-aa21-46d7-897e-5da399001fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:30.798402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19c6382-aa21-46d7-897e-5da399001fd0 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:30.799044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c6382-aa21-46d7-897e-5da399001fd0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:30.799330 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c6382-aa21-46d7-897e-5da399001fd0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:30.800772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19c6382-aa21-46d7-897e-5da399001fd0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:30.813902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 623/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 14:01:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:30.820251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:30.920153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad7ea6-cc75-4fc6-b334-146de9a72694 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:30.954273 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbed46-a198-498e-b103-78e1a8885885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:30.954873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbed46-a198-498e-b103-78e1a8885885 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:30.955143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbed46-a198-498e-b103-78e1a8885885 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:30.955405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbed46-a198-498e-b103-78e1a8885885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:30.956120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbed46-a198-498e-b103-78e1a8885885 None admin] RBAC: Adding query filter params (role.id=7d39e08123cb40c78586e4d7eb3ed9ba, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=5b9792d95cad4f0d8708df0a608f30ed) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:30.956421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbed46-a198-498e-b103-78e1a8885885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:30.957865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbed46-a198-498e-b103-78e1a8885885 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:30.963782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 623/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 14:01:30 2022] GET /identity/v3/role_assignments?user.id=5b9792d95cad4f0d8708df0a608f30ed&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=7d39e08123cb40c78586e4d7eb3ed9ba => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:33.576685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4fcdd-3a2e-4331-ad75-fe0771ac6611 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:33.577256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4fcdd-3a2e-4331-ad75-fe0771ac6611 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:33.577256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4fcdd-3a2e-4331-ad75-fe0771ac6611 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:33.578589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 624/1247] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:33.584079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b3199-1a40-488e-a53a-3bc6740cb428 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:33.584557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b3199-1a40-488e-a53a-3bc6740cb428 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:33.584663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b3199-1a40-488e-a53a-3bc6740cb428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:33.591912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:33.605366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6c1b3199-1a40-488e-a53a-3bc6740cb428 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:33.606647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c1b3199-1a40-488e-a53a-3bc6740cb428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:33.682378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 624/1248] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:33.943947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4fcdd-3a2e-4331-ad75-fe0771ac6611 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:33.965810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:34.058609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4fcdd-3a2e-4331-ad75-fe0771ac6611 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:34.091096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3ca43-ae1a-4e22-b868-a235eca52489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:34.091728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3ca43-ae1a-4e22-b868-a235eca52489 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:34.092040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3ca43-ae1a-4e22-b868-a235eca52489 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:34.092308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc3ca43-ae1a-4e22-b868-a235eca52489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:34.102345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc3ca43-ae1a-4e22-b868-a235eca52489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-9fc3ca43-ae1a-4e22-b868-a235eca52489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:01:34.107035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:34.108707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:34.109963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 625/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 14:01:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:34.114901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b3199-1a40-488e-a53a-3bc6740cb428 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:34.243954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b3199-1a40-488e-a53a-3bc6740cb428 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:34.282451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:34.288530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:34.289421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:34.289784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:34.290027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:34.290748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:34.291096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:34.292515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:34.301363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 625/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 14:01:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:34.307529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3ca43-ae1a-4e22-b868-a235eca52489 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:34.393305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:34.420447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc3ca43-ae1a-4e22-b868-a235eca52489 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:34.465867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:34.466732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:34.467212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:34.467681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:34.480300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:34.481790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:34.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:34.497692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:34.498785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:34.498785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:34.498785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:34.498785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 626/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 05 14:01:34.504516 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:34.614012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92b76a5-8603-4f89-af8a-fece007bf07a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:34.649896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:34.650523 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:34.650795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:34.651089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:34.651697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:34.652004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:34.653348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:34.665717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 626/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:34.671828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:34.789406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366fae7c-8c3b-4309-92d7-7d18e2aa3060 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:34.817083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:34.826192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:34.827150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:34.827641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:34.827972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:34.837749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:34.839502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 14:01:34.852870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:34.854168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:34.855551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 627/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:34.860993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:34.922873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:34.971178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab5149-2fab-43de-9d1c-fd364a21f136 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:35.006553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:35.007172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:35.007447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:35.007715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:35.008742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:35.009073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:35.010688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:35.030811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 627/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 14:01:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:35.037038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:35.143440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291dfb55-a6ff-4fae-a6f1-f9153f3e453d None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:35.182087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b2e83-3d54-4875-a4c4-16a0900e94fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:35.183041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2e83-3d54-4875-a4c4-16a0900e94fe None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:35.183405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2e83-3d54-4875-a4c4-16a0900e94fe None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:35.183817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26b2e83-3d54-4875-a4c4-16a0900e94fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:35.256113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:35.272155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b2e83-3d54-4875-a4c4-16a0900e94fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:35.273854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26b2e83-3d54-4875-a4c4-16a0900e94fe None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:35.375790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 628/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 14:01:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 14:01:35.382064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:35.488761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cd3a4a-256b-43e1-9ce7-c5c49be3c139 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:35.502856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:35.531040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:35.531685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:35.531964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:35.532236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:35.545499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:35.547061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:35.571080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:35.572485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:35.573564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:35.573564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:35.573564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:35.573564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 628/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:35.579715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b2e83-3d54-4875-a4c4-16a0900e94fe None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:35.677067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:35.691149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26b2e83-3d54-4875-a4c4-16a0900e94fe None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:35.725957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:35.726581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:35.726852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:35.727114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:35.727762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:35.728055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:35.729456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:35.743171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 629/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:35.749402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:35.860233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c95fa-2b58-4e7d-93fc-38265c87db71 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:35.902572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:35.903225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:35.903501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:35.903771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:35.911610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:35.913055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 14:01:35.925505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:35.931564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:35.932785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 629/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 14:01:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:35.934915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:36.040791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fab43-d287-4ba8-b1cd-fcc3398a530f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:36.074507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86303a-2317-4a80-bb47-16732b33aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:36.075115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86303a-2317-4a80-bb47-16732b33aed2 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:36.075382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86303a-2317-4a80-bb47-16732b33aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:36.075647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86303a-2317-4a80-bb47-16732b33aed2 None admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:36.076360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86303a-2317-4a80-bb47-16732b33aed2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:36.076662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86303a-2317-4a80-bb47-16732b33aed2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:36.078142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86303a-2317-4a80-bb47-16732b33aed2 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:36.090922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 630/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 14:01:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:36.097468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:36.125238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:36.214103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb7d79-91dd-446c-9bd0-7f8c1d2e203a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:36.250993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5b7f99-3251-4365-a5a0-380bd0d8f1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:36.251608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b7f99-3251-4365-a5a0-380bd0d8f1e0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:36.253626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b7f99-3251-4365-a5a0-380bd0d8f1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:36.253626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5b7f99-3251-4365-a5a0-380bd0d8f1e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:36.253626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b7f99-3251-4365-a5a0-380bd0d8f1e0 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=5b9792d95cad4f0d8708df0a608f30ed) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:36.253626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b7f99-3251-4365-a5a0-380bd0d8f1e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:36.254647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5b7f99-3251-4365-a5a0-380bd0d8f1e0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:36.260741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 630/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 14:01:36 2022] GET /identity/v3/role_assignments?user.id=5b9792d95cad4f0d8708df0a608f30ed&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:38.343792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998385f9-676b-44d7-99e2-e95a9366c07a None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:38.344388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998385f9-676b-44d7-99e2-e95a9366c07a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:38.344388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998385f9-676b-44d7-99e2-e95a9366c07a None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:38.345735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 631/1261] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:38.351303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7bad-6585-404c-9a7b-79735ef9e57a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:38.351728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7bad-6585-404c-9a7b-79735ef9e57a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:38.351836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d7bad-6585-404c-9a7b-79735ef9e57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:38.372056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3c2d7bad-6585-404c-9a7b-79735ef9e57a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:38.373286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3c2d7bad-6585-404c-9a7b-79735ef9e57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:38.444751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 631/1262] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:38.713549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998385f9-676b-44d7-99e2-e95a9366c07a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:38.732543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:38.828094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998385f9-676b-44d7-99e2-e95a9366c07a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:38.865033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf09de3-916d-4e31-90bf-6dd7ec1ba2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:38.865673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf09de3-916d-4e31-90bf-6dd7ec1ba2da None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:38.865948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf09de3-916d-4e31-90bf-6dd7ec1ba2da None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:38.866218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf09de3-916d-4e31-90bf-6dd7ec1ba2da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:38.876926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf09de3-916d-4e31-90bf-6dd7ec1ba2da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-8cf09de3-916d-4e31-90bf-6dd7ec1ba2da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:01:38.881671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:38.882730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:38.883927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 632/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 14:01:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:38.889128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d7bad-6585-404c-9a7b-79735ef9e57a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:38.907607 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:38.992322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d7bad-6585-404c-9a7b-79735ef9e57a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:39.029556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:39.030187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:39.030463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:39.030730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:39.031389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:39.031694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:39.033062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:39.041088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 632/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 14:01:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:39.047517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf09de3-916d-4e31-90bf-6dd7ec1ba2da None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:39.123697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:39.160379 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf09de3-916d-4e31-90bf-6dd7ec1ba2da None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:39.197690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:39.198329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:39.198599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:39.198871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:39.209712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:39.211393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:39.227943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:39.229159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:39.230084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:39.230084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:39.230084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:39.230084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 633/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:39.235774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:39.319881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:39.340851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb1597-327b-4795-acf7-3762b62b4abc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:39.376762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:39.377461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:39.377702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:39.377924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:39.378592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:39.378893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:39.380278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:39.392260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 633/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:39.400777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:39.507164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dec9d2-0b6f-4cf3-b34a-866fc838dac8 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:39.529276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:39.544911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:39.545546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:39.545852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:39.546111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:39.553871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:39.555307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 14:01:39.566841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:39.571685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:39.572919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 634/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:39.574006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:39.678967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31eb4fe-c4e6-4f37-a743-8f883d0f4d1f None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:39.721051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:39.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:39.721976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:39.722246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:39.723147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:39.723459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:39.724888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:39.734327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:39.743054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 634/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 14:01:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:39.749858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:39.871882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab854c66-538c-4c9d-88b1-da870ad6e9ae None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:39.907424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028facd-a590-4c63-96f1-38747335372a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:39.908042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028facd-a590-4c63-96f1-38747335372a None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:39.908319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028facd-a590-4c63-96f1-38747335372a None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:39.908588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4028facd-a590-4c63-96f1-38747335372a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:39.919488 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028facd-a590-4c63-96f1-38747335372a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:39.920941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4028facd-a590-4c63-96f1-38747335372a None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-4028facd-a590-4c63-96f1-38747335372a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:39.935358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:39.936400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:39.937327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:39.937327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:39.937327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:39.937327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 635/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:39.944555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:40.048965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9602e980-588d-44a0-bfdc-32b1ab3f7e83 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:40.084406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:40.085001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:40.085274 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:40.085546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:40.086224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:40.086529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:40.087895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:40.099157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 635/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:40.105512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028facd-a590-4c63-96f1-38747335372a None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:40.142872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:40.216570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4028facd-a590-4c63-96f1-38747335372a None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:40.254001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:40.254642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:40.254918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:40.255191 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:40.262094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:40.263637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 14:01:40.275754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:40.276658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:40.277651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 636/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 14:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:40.285732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:40.348493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:40.392152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35bfbd-5798-47e5-bd2d-e8981fa00b8d None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:40.425301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda7fa23-0df2-4f00-83bd-1342c00e8d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:40.425914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7fa23-0df2-4f00-83bd-1342c00e8d2e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:40.426188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7fa23-0df2-4f00-83bd-1342c00e8d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:40.426453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda7fa23-0df2-4f00-83bd-1342c00e8d2e None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:40.427127 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7fa23-0df2-4f00-83bd-1342c00e8d2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:40.427430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7fa23-0df2-4f00-83bd-1342c00e8d2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:40.428854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda7fa23-0df2-4f00-83bd-1342c00e8d2e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:40.441382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 636/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 14:01:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:40.448422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:40.549316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:40.558398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f488ba7-263c-49f4-a528-921dd65b7975 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:40.592167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1345976d-c3f5-425f-bc5e-a4f20f12c76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:40.592839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345976d-c3f5-425f-bc5e-a4f20f12c76f None admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:40.593108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345976d-c3f5-425f-bc5e-a4f20f12c76f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:40.593374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1345976d-c3f5-425f-bc5e-a4f20f12c76f None admin] PATH_INFO: `/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/5b9792d95cad4f0d8708df0a608f30ed/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:40.595547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345976d-c3f5-425f-bc5e-a4f20f12c76f None admin] RBAC: Authorizing `identity:create_grant(project_id=badaa5d0952a4dfaa32a5f179213b689, user_id=5b9792d95cad4f0d8708df0a608f30ed, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:40.596981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1345976d-c3f5-425f-bc5e-a4f20f12c76f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:40.616201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 637/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Wed Oct 5 14:01:40 2022] PUT /identity/v3/projects/badaa5d0952a4dfaa32a5f179213b689/users/5b9792d95cad4f0d8708df0a608f30ed/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:01:42.721714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550ac1-2810-4616-b714-b49a37438850 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:42.722587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550ac1-2810-4616-b714-b49a37438850 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:42.722587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9550ac1-2810-4616-b714-b49a37438850 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:42.724834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 637/1274] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:42.731102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67397d96-72c1-40ca-a563-9d8f9d979269 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:42.731532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67397d96-72c1-40ca-a563-9d8f9d979269 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:42.731643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67397d96-72c1-40ca-a563-9d8f9d979269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:42.757261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-67397d96-72c1-40ca-a563-9d8f9d979269 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:42.758563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-67397d96-72c1-40ca-a563-9d8f9d979269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:42.827472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 638/1275] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:43.103242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9550ac1-2810-4616-b714-b49a37438850 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:43.239108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9550ac1-2810-4616-b714-b49a37438850 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:43.262522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:43.282656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c62c2-12f6-4f39-adfc-f5ade18ce4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:43.283296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c62c2-12f6-4f39-adfc-f5ade18ce4bc None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:43.283570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c62c2-12f6-4f39-adfc-f5ade18ce4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:43.283838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c62c2-12f6-4f39-adfc-f5ade18ce4bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:43.294149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c62c2-12f6-4f39-adfc-f5ade18ce4bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-7e1c62c2-12f6-4f39-adfc-f5ade18ce4bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 05 14:01:43.298949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:43.300083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:43.301617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 638/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Wed Oct 5 14:01:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:43.308896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67397d96-72c1-40ca-a563-9d8f9d979269 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:43.393078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:43.421862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67397d96-72c1-40ca-a563-9d8f9d979269 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:43.456676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:43.457276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:43.457552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:43.457849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:43.458503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:43.458795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:43.460171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:43.468259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 639/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Wed Oct 5 14:01:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:43.474983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c62c2-12f6-4f39-adfc-f5ade18ce4bc None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:43.591830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c62c2-12f6-4f39-adfc-f5ade18ce4bc None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:43.631389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:43.632033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:43.632301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:43.632564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:43.646818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:43.649056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:43.670066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:43.671192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:43.672101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:43.672101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:43.672101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:43.672101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 639/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:43.677451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:43.787990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418c33-23b2-4fe3-8fd2-280b62ce0c18 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:43.822090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:43.822716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:43.822999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:43.823265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:43.823907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:43.824187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:43.825596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:43.837177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 640/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:43.843792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:43.887052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:43.959111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce0729b-e8fc-4ae7-984f-15d6964e1a56 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:44.007190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:44.007841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:44.008117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:44.008385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:44.016545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:44.018012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 05 14:01:44.029089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:44.030170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:44.031262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 640/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:44.038249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:44.042282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:44.141229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf0931b-c293-403a-8da5-744f388600e3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:44.179270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:44.180004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:44.180278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:44.180561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:44.181370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:44.181729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:44.183176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:44.201740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 641/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Wed Oct 5 14:01:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:44.207907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:44.311268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:44.327631 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55000e6c-e66d-488c-bd42-8ab37c5b0984 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:44.368657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1c9bc-a94b-4d05-812e-1d8ca88952e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:44.369250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1c9bc-a94b-4d05-812e-1d8ca88952e6 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:44.369520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1c9bc-a94b-4d05-812e-1d8ca88952e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:44.369831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1c9bc-a94b-4d05-812e-1d8ca88952e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:44.439410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1c9bc-a94b-4d05-812e-1d8ca88952e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:44.440854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1c9bc-a94b-4d05-812e-1d8ca88952e6 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:44.535671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:44.546490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 641/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Wed Oct 5 14:01:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 05 14:01:44.552422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:44.615382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:44.659731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e6777-2b96-4e86-aabc-4e1a409d034f None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:44.694378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:44.694996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:44.695272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:44.695540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:44.705671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:44.707226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 05 14:01:44.721668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:44.722713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 05 14:01:44.723994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 05 14:01:44.723994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 05 14:01:44.723994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:44.723994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 642/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:44.731369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1c9bc-a94b-4d05-812e-1d8ca88952e6 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:44.830110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1c9bc-a94b-4d05-812e-1d8ca88952e6 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:44.865501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:44.866105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:44.866320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:44.866582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:44.867214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:44.867509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:44.868866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:44.881439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 642/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Wed Oct 5 14:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:44.887505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:44.985423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb718e6-ff0d-47d0-bb80-758b289431fa None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:44.999052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:45.020980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:45.021604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:45.021907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:45.022181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:45.029186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:45.030713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 05 14:01:45.042110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:45.043141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:01:45.044016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 643/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Wed Oct 5 14:01:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:45.049541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:45.099588 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:45.151428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfbd00e-059a-4511-ae6e-35725b06d933 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:45.191828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69d6fd1-c034-4ea7-88d9-86a321e6858e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:45.192476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d6fd1-c034-4ea7-88d9-86a321e6858e None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:45.192747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d6fd1-c034-4ea7-88d9-86a321e6858e None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:45.193016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69d6fd1-c034-4ea7-88d9-86a321e6858e None admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:45.193743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d6fd1-c034-4ea7-88d9-86a321e6858e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:45.194046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d6fd1-c034-4ea7-88d9-86a321e6858e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:45.195484 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69d6fd1-c034-4ea7-88d9-86a321e6858e None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:45.208018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 643/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Wed Oct 5 14:01:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:45.214616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:45.323643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37386386-7059-4646-a9c2-ca589ceb21b3 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:45.358373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2037d64-4437-4d34-9a23-49f0f7040cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:45.359054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2037d64-4437-4d34-9a23-49f0f7040cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:45.359325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2037d64-4437-4d34-9a23-49f0f7040cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:45.359595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2037d64-4437-4d34-9a23-49f0f7040cb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:45.360316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2037d64-4437-4d34-9a23-49f0f7040cb0 None admin] RBAC: Adding query filter params (role.id=b9327f2ca9284c5ea652cbce8217b47d, scope.project.id=badaa5d0952a4dfaa32a5f179213b689, user.id=5b9792d95cad4f0d8708df0a608f30ed) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:45.360624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2037d64-4437-4d34-9a23-49f0f7040cb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:45.362082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2037d64-4437-4d34-9a23-49f0f7040cb0 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:45.367886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 644/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Wed Oct 5 14:01:45 2022] GET /identity/v3/role_assignments?user.id=5b9792d95cad4f0d8708df0a608f30ed&scope.project.id=badaa5d0952a4dfaa32a5f179213b689&role.id=b9327f2ca9284c5ea652cbce8217b47d => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:47.491083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84e783-6db0-4f78-b4e7-3cd243b057af None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:47.491941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84e783-6db0-4f78-b4e7-3cd243b057af None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:47.491941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b84e783-6db0-4f78-b4e7-3cd243b057af None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:47.494095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 644/1288] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:47.499494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea7a18-d9b9-488e-83c0-79169ad21c68 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:47.499933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea7a18-d9b9-488e-83c0-79169ad21c68 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:47.500025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea7a18-d9b9-488e-83c0-79169ad21c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:47.519633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8aea7a18-d9b9-488e-83c0-79169ad21c68 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:47.520808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8aea7a18-d9b9-488e-83c0-79169ad21c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:47.552604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:47.592479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 645/1289] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:47.869919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84e783-6db0-4f78-b4e7-3cd243b057af None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:47.983609 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b84e783-6db0-4f78-b4e7-3cd243b057af None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:47.994221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:48.020037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:48.020653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:48.020923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:48.021188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:48.021790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:48.023150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:48.031848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:48.033019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:48.034130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:48.037724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 645/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 14:01:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:48.038904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea7a18-d9b9-488e-83c0-79169ad21c68 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:48.149309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea7a18-d9b9-488e-83c0-79169ad21c68 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:48.189756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d862c1f4-d287-4493-8f94-756207bc51e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:48.190463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862c1f4-d287-4493-8f94-756207bc51e9 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:48.190766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862c1f4-d287-4493-8f94-756207bc51e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:48.191029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d862c1f4-d287-4493-8f94-756207bc51e9 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:48.191701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862c1f4-d287-4493-8f94-756207bc51e9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:48.191951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862c1f4-d287-4493-8f94-756207bc51e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:48.193395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d862c1f4-d287-4493-8f94-756207bc51e9 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:48.198835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 646/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Oct 5 14:01:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:01:48.205249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:48.318319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ffaaf-076f-4922-9834-e3bebe37f61a None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:48.356206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b980150-7129-4aed-875b-3cced6ce5c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:48.357133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980150-7129-4aed-875b-3cced6ce5c08 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:48.357403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980150-7129-4aed-875b-3cced6ce5c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:48.357679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b980150-7129-4aed-875b-3cced6ce5c08 None admin] PATH_INFO: `/v3/services` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:48.358367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b980150-7129-4aed-875b-3cced6ce5c08 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:48.358660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b980150-7129-4aed-875b-3cced6ce5c08 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:48.360017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b980150-7129-4aed-875b-3cced6ce5c08 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:48.364993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 646/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Oct 5 14:01:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:50.474899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c928-ecd7-40e3-be46-379db2957fdf None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:50.475470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c928-ecd7-40e3-be46-379db2957fdf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:50.475470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b7c928-ecd7-40e3-be46-379db2957fdf None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:50.476815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 647/1293] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:50.483873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310bcc-7cf5-4d40-9d85-f205deb1fab8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:50.484596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310bcc-7cf5-4d40-9d85-f205deb1fab8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:50.484722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310bcc-7cf5-4d40-9d85-f205deb1fab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:50.509703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b5310bcc-7cf5-4d40-9d85-f205deb1fab8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:50.510929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b5310bcc-7cf5-4d40-9d85-f205deb1fab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:50.542023 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:50.585131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 647/1294] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:50.847890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7c928-ecd7-40e3-be46-379db2957fdf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:50.885062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:50.965227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b7c928-ecd7-40e3-be46-379db2957fdf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:51.000157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2263612-325b-4854-9b47-279bcd66e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:51.000755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263612-325b-4854-9b47-279bcd66e8ce None admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:51.001025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263612-325b-4854-9b47-279bcd66e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:51.001290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2263612-325b-4854-9b47-279bcd66e8ce None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:51.002006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2263612-325b-4854-9b47-279bcd66e8ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:51.003403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2263612-325b-4854-9b47-279bcd66e8ce None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:51.023082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 648/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Wed Oct 5 14:01:50 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:01:53.166109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aec2df-66ba-47c4-8f99-23c5bef796c0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:53.167024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aec2df-66ba-47c4-8f99-23c5bef796c0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:53.167024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aec2df-66ba-47c4-8f99-23c5bef796c0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:53.169088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 648/1296] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:53.175093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2edc7a-3b7f-4093-95fa-4ec894b6ca39 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:53.176044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2edc7a-3b7f-4093-95fa-4ec894b6ca39 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:53.176359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2edc7a-3b7f-4093-95fa-4ec894b6ca39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:53.207684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bc2edc7a-3b7f-4093-95fa-4ec894b6ca39 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:53.209371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc2edc7a-3b7f-4093-95fa-4ec894b6ca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:53.292494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 649/1297] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 05 14:01:53.554464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aec2df-66ba-47c4-8f99-23c5bef796c0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:53.666275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aec2df-66ba-47c4-8f99-23c5bef796c0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:53.704923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:53.705555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:53.705865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:53.706129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:53.706708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:53.708104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:53.719345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:53.720614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:53.721717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 649/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 14:01:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:53.726687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2edc7a-3b7f-4093-95fa-4ec894b6ca39 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:53.745443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:53.842706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2edc7a-3b7f-4093-95fa-4ec894b6ca39 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:53.885913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:53.886807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:53.887175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:53.887532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:53.888513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:53.888906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:53.891028 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:53.901196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 650/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Oct 5 14:01:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:53.907545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:53.940175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:54.010344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f0496c-a004-4b64-977e-afd27855aca4 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:54.052629 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1a7c55-41b4-4ac5-ada8-57a71ce1f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:54.053215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7c55-41b4-4ac5-ada8-57a71ce1f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:54.053487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7c55-41b4-4ac5-ada8-57a71ce1f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:54.053786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1a7c55-41b4-4ac5-ada8-57a71ce1f7e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:54.054478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a7c55-41b4-4ac5-ada8-57a71ce1f7e0 None admin] RBAC: Adding query filter params (interface=public, service_id=1064972edf5e4f9baf540339dec66a37, region_id=RegionOne) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:54.054781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a7c55-41b4-4ac5-ada8-57a71ce1f7e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:54.056139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1a7c55-41b4-4ac5-ada8-57a71ce1f7e0 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:54.061319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 650/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Wed Oct 5 14:01:53 2022] GET /identity/v3/endpoints?service_id=1064972edf5e4f9baf540339dec66a37&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:54.070532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:54.167891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:54.188823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff3a0c-26cf-40b3-9dc8-001faf275317 None admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:54.224397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde6625f-1fec-4a52-8128-81e01db181b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:54.225004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6625f-1fec-4a52-8128-81e01db181b7 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:54.225281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6625f-1fec-4a52-8128-81e01db181b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:54.225549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6625f-1fec-4a52-8128-81e01db181b7 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:54.226203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6625f-1fec-4a52-8128-81e01db181b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:54.227584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde6625f-1fec-4a52-8128-81e01db181b7 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:54.234940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 651/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Wed Oct 5 14:01:54 2022] GET /identity/v3/services => generated 2191 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:01:56.349121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae35b8-bc24-4f0b-a863-1f07e326442d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:56.350137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae35b8-bc24-4f0b-a863-1f07e326442d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:56.350137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae35b8-bc24-4f0b-a863-1f07e326442d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:56.352236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 651/1302] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:01:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:01:56.359091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5067c0-a096-46b8-bc74-797cad556749 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:56.359836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5067c0-a096-46b8-bc74-797cad556749 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:56.359959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5067c0-a096-46b8-bc74-797cad556749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:56.384160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-db5067c0-a096-46b8-bc74-797cad556749 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:01:56.385354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-db5067c0-a096-46b8-bc74-797cad556749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:01:56.453390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 652/1303] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:01:56 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 05 14:01:56.720981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae35b8-bc24-4f0b-a863-1f07e326442d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:56.825134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:56.832530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae35b8-bc24-4f0b-a863-1f07e326442d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:56.866960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:56.867590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:56.867864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:56.868134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:56.868708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:56.870107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:01:56.879275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:01:56.880365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:01:56.881256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 652/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Wed Oct 5 14:01:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 05 14:01:56.887406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5067c0-a096-46b8-bc74-797cad556749 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:56.966967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:01:56.994936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5067c0-a096-46b8-bc74-797cad556749 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:57.030752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c22901-948c-44c3-a302-ce9973c95e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:57.031356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c22901-948c-44c3-a302-ce9973c95e79 None admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:57.031633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c22901-948c-44c3-a302-ce9973c95e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:57.031902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c22901-948c-44c3-a302-ce9973c95e79 None admin] PATH_INFO: `/v3/services` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:57.032524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c22901-948c-44c3-a302-ce9973c95e79 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:01:57.032823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c22901-948c-44c3-a302-ce9973c95e79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:57.034229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c22901-948c-44c3-a302-ce9973c95e79 None admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:57.039360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 653/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Wed Oct 5 14:01:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:01:57.048555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:01:57.155665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e7e41b-a53a-423e-b448-80439a3ec243 None admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:01:57.197177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5195cfcf-9d54-4526-b6a5-55b8811c1177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:01:57.197938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195cfcf-9d54-4526-b6a5-55b8811c1177 None admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:01:57.198363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195cfcf-9d54-4526-b6a5-55b8811c1177 None admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:01:57.198669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5195cfcf-9d54-4526-b6a5-55b8811c1177 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:01:57.199394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195cfcf-9d54-4526-b6a5-55b8811c1177 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:01:57.201199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5195cfcf-9d54-4526-b6a5-55b8811c1177 None admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:01:57.223985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 653/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:01:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:02:07.036286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=30 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:02:07.219316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:01.780886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbdb4b-316c-4789-82f1-57127d802ba3 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:01.781460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbdb4b-316c-4789-82f1-57127d802ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:01.781460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecbdb4b-316c-4789-82f1-57127d802ba3 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:01.786016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 654/1307] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:01.788133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705dec6-155f-497c-8c93-9f93f69444ca None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:01.788561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705dec6-155f-497c-8c93-9f93f69444ca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:01.788657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705dec6-155f-497c-8c93-9f93f69444ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:01.944292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c705dec6-155f-497c-8c93-9f93f69444ca None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:01.945706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c705dec6-155f-497c-8c93-9f93f69444ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:02.190207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 654/1308] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:02.465515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c23fb-694b-43b8-bdec-094fdd50084d None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:02.466129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c23fb-694b-43b8-bdec-094fdd50084d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:02.466564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c23fb-694b-43b8-bdec-094fdd50084d None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:02.468273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 655/1309] 10.222.0.23 () {54 vars in 964 bytes} [Wed Oct 5 14:03:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:02.475374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa452-641a-45cf-b4d8-54dfb304bea7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:02.475813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa452-641a-45cf-b4d8-54dfb304bea7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:02.476201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0daa452-641a-45cf-b4d8-54dfb304bea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:02.598691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e0daa452-641a-45cf-b4d8-54dfb304bea7 None None] MFA Rules not processed for user `6c9b940cd9e0432daa235f82becb4d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:02.600079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e0daa452-641a-45cf-b4d8-54dfb304bea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:02.627766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:02.696993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 655/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Wed Oct 5 14:03:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:02.707433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c23fb-694b-43b8-bdec-094fdd50084d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:02.709357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-518c23fb-694b-43b8-bdec-094fdd50084d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:02.827926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c23fb-694b-43b8-bdec-094fdd50084d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:02.908952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc80846d-e7f2-42e6-b51b-ec59a25dab1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:02.909719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80846d-e7f2-42e6-b51b-ec59a25dab1e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:02.910061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80846d-e7f2-42e6-b51b-ec59a25dab1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:02.910387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc80846d-e7f2-42e6-b51b-ec59a25dab1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:02.912094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc80846d-e7f2-42e6-b51b-ec59a25dab1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:02.972327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:03.030448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80846d-e7f2-42e6-b51b-ec59a25dab1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:03.031947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc80846d-e7f2-42e6-b51b-ec59a25dab1e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:03.051616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 656/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:05.507002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965b53-6e8b-4db2-9d4c-1a2d4691f48e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:05.507493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965b53-6e8b-4db2-9d4c-1a2d4691f48e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:05.507697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd965b53-6e8b-4db2-9d4c-1a2d4691f48e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:05.508908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 656/1312] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:05.514600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26554-62eb-441b-9bba-a978f6b070ba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:05.515042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26554-62eb-441b-9bba-a978f6b070ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:05.515137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c26554-62eb-441b-9bba-a978f6b070ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:05.586023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d5c26554-62eb-441b-9bba-a978f6b070ba None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:05.587402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d5c26554-62eb-441b-9bba-a978f6b070ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:05.698234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 657/1313] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:05.980642 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd965b53-6e8b-4db2-9d4c-1a2d4691f48e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:05.982236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cd965b53-6e8b-4db2-9d4c-1a2d4691f48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:06.091236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd965b53-6e8b-4db2-9d4c-1a2d4691f48e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:06.166401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:06.168593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3113-771b-4919-9455-281997f14d3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:06.169195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3113-771b-4919-9455-281997f14d3b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:06.169467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3113-771b-4919-9455-281997f14d3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:06.169821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3113-771b-4919-9455-281997f14d3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:06.171955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-711a3113-771b-4919-9455-281997f14d3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:06.284142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a3113-771b-4919-9455-281997f14d3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:06.285514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711a3113-771b-4919-9455-281997f14d3b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:06.308834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 657/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:03:08.779229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dff3f7-da9c-4820-8add-dda5588b1216 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:08.779796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dff3f7-da9c-4820-8add-dda5588b1216 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:08.779796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dff3f7-da9c-4820-8add-dda5588b1216 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:08.781047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 658/1315] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:08.787642 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca70f-6454-4bab-8fd9-f09b303c4c81 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:08.788363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca70f-6454-4bab-8fd9-f09b303c4c81 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:08.788509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ca70f-6454-4bab-8fd9-f09b303c4c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:08.859126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-274ca70f-6454-4bab-8fd9-f09b303c4c81 None None] MFA Rules not processed for user `66a9447c53c74350bc5f1a7039895a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:08.860346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-274ca70f-6454-4bab-8fd9-f09b303c4c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:08.915498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:08.964353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 658/1316] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:08 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:09.255013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dff3f7-da9c-4820-8add-dda5588b1216 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:09.257497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-99dff3f7-da9c-4820-8add-dda5588b1216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:09.270713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:09.371114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dff3f7-da9c-4820-8add-dda5588b1216 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:09.450533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc88712d-8a43-4915-bf3e-4ecbcfa51b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:09.451189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88712d-8a43-4915-bf3e-4ecbcfa51b1c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:09.451473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88712d-8a43-4915-bf3e-4ecbcfa51b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:09.451747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc88712d-8a43-4915-bf3e-4ecbcfa51b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:09.453721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cc88712d-8a43-4915-bf3e-4ecbcfa51b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:09.524079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:09.577307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88712d-8a43-4915-bf3e-4ecbcfa51b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:09.578809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc88712d-8a43-4915-bf3e-4ecbcfa51b1c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:09.597151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 659/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:09 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:12.277031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c0e8f-1503-484f-8333-98fc1faf15e6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:12.277597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c0e8f-1503-484f-8333-98fc1faf15e6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:12.277597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c0e8f-1503-484f-8333-98fc1faf15e6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:12.278953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 659/1318] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:12.285453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d371b-9a23-49ff-b139-2d304a7819d9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:12.286260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d371b-9a23-49ff-b139-2d304a7819d9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:12.286395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686d371b-9a23-49ff-b139-2d304a7819d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:12.367105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-686d371b-9a23-49ff-b139-2d304a7819d9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:12.368352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-686d371b-9a23-49ff-b139-2d304a7819d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:12.469207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 660/1319] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:12.763585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55db8b-c567-45f2-b1b1-5748b780a9ea None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:12.763936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55db8b-c567-45f2-b1b1-5748b780a9ea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:12.764233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55db8b-c567-45f2-b1b1-5748b780a9ea None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:12.766365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 660/1320] 10.222.0.23 () {54 vars in 964 bytes} [Wed Oct 5 14:03:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:12.772880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b91e4-506d-4395-965c-d86eb28cf298 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:12.773278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b91e4-506d-4395-965c-d86eb28cf298 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:12.773673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b91e4-506d-4395-965c-d86eb28cf298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:12.799336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:12.858967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3b8b91e4-506d-4395-965c-d86eb28cf298 None None] MFA Rules not processed for user `6c9b940cd9e0432daa235f82becb4d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:12.860175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3b8b91e4-506d-4395-965c-d86eb28cf298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:12.941197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 661/1321] 10.222.0.23 () {58 vars in 1061 bytes} [Wed Oct 5 14:03:12 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:12.950797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55db8b-c567-45f2-b1b1-5748b780a9ea None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:12.953141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0c55db8b-c567-45f2-b1b1-5748b780a9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:13.066414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55db8b-c567-45f2-b1b1-5748b780a9ea None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:13.144858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0781e953-0cd5-460b-9327-5fe4c43b254e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:13.145469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781e953-0cd5-460b-9327-5fe4c43b254e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:13.145787 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781e953-0cd5-460b-9327-5fe4c43b254e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:13.146061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0781e953-0cd5-460b-9327-5fe4c43b254e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:13.147656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0781e953-0cd5-460b-9327-5fe4c43b254e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:13.154830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:13.273181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781e953-0cd5-460b-9327-5fe4c43b254e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:13.274655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0781e953-0cd5-460b-9327-5fe4c43b254e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:13.293142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 661/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:16.578694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1d81-dbc7-4a4f-86de-929500b3190f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:16.579279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1d81-dbc7-4a4f-86de-929500b3190f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:16.579279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdd1d81-dbc7-4a4f-86de-929500b3190f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:16.580571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 662/1323] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:16.586214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2144f-b150-41f4-96fb-d428447a36cc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:16.586689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2144f-b150-41f4-96fb-d428447a36cc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:16.586778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2144f-b150-41f4-96fb-d428447a36cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:16.665264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f8d2144f-b150-41f4-96fb-d428447a36cc None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:16.666524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f8d2144f-b150-41f4-96fb-d428447a36cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:16.733700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:16.776907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 662/1324] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:17.055228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd1d81-dbc7-4a4f-86de-929500b3190f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:17.056758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1bdd1d81-dbc7-4a4f-86de-929500b3190f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:17.164901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdd1d81-dbc7-4a4f-86de-929500b3190f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:17.179019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:17.244460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646f969-8974-4e8c-a09f-21366a368a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:17.245091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f969-8974-4e8c-a09f-21366a368a5b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:17.245390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f969-8974-4e8c-a09f-21366a368a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:17.245655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646f969-8974-4e8c-a09f-21366a368a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:17.247282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c646f969-8974-4e8c-a09f-21366a368a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:17.362000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646f969-8974-4e8c-a09f-21366a368a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:17.363366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646f969-8974-4e8c-a09f-21366a368a5b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:17.381433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 663/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:19.562464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ed74a-1974-4415-b1c9-6855a3bf1fa4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:19.563049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ed74a-1974-4415-b1c9-6855a3bf1fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:19.563049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ed74a-1974-4415-b1c9-6855a3bf1fa4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:19.564362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 663/1326] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:19.569919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ce915-f8dd-495d-8cac-5da8a556d854 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:19.570378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ce915-f8dd-495d-8cac-5da8a556d854 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:19.570568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7ce915-f8dd-495d-8cac-5da8a556d854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:19.621701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:19.640607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1e7ce915-f8dd-495d-8cac-5da8a556d854 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:19.641888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1e7ce915-f8dd-495d-8cac-5da8a556d854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:19.746344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 664/1327] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:20.021979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ed74a-1974-4415-b1c9-6855a3bf1fa4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:20.023523 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2c9ed74a-1974-4415-b1c9-6855a3bf1fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:20.132305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ed74a-1974-4415-b1c9-6855a3bf1fa4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:20.211450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e053bd-e50c-4136-81f8-6d29a004f02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:20.212053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e053bd-e50c-4136-81f8-6d29a004f02f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:20.212327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e053bd-e50c-4136-81f8-6d29a004f02f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:20.212587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e053bd-e50c-4136-81f8-6d29a004f02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:20.214214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-63e053bd-e50c-4136-81f8-6d29a004f02f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:20.232721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:20.334085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e053bd-e50c-4136-81f8-6d29a004f02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:20.335438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e053bd-e50c-4136-81f8-6d29a004f02f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:20.355702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 664/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:22.575729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544312d-8c17-4153-801b-b297eb276ed0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:22.576566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544312d-8c17-4153-801b-b297eb276ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:22.576566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544312d-8c17-4153-801b-b297eb276ed0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:22.578624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 665/1329] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:22.586805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3cb14-bbc9-4738-aad0-2a7375a4716f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:22.587280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3cb14-bbc9-4738-aad0-2a7375a4716f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:22.587395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3cb14-bbc9-4738-aad0-2a7375a4716f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:22.656551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5dc3cb14-bbc9-4738-aad0-2a7375a4716f None None] MFA Rules not processed for user `66a9447c53c74350bc5f1a7039895a77`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:22.657798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5dc3cb14-bbc9-4738-aad0-2a7375a4716f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:22.739729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:22.772647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 665/1330] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:22 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:23.049927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544312d-8c17-4153-801b-b297eb276ed0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:23.051501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a544312d-8c17-4153-801b-b297eb276ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:23.159768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544312d-8c17-4153-801b-b297eb276ed0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:23.182266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:23.240519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a3a8d-e4d8-4d96-b9db-83b9d884c96a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:23.241129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a3a8d-e4d8-4d96-b9db-83b9d884c96a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:23.241403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a3a8d-e4d8-4d96-b9db-83b9d884c96a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:23.241662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a3a8d-e4d8-4d96-b9db-83b9d884c96a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:23.243283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a62a3a8d-e4d8-4d96-b9db-83b9d884c96a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:23.356204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a3a8d-e4d8-4d96-b9db-83b9d884c96a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:23.357677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a3a8d-e4d8-4d96-b9db-83b9d884c96a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:23.376068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 666/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:23 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:25.745811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d129761-d0e4-46f9-bc85-d2d9ea5f9d13 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:25.746614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d129761-d0e4-46f9-bc85-d2d9ea5f9d13 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:25.746614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d129761-d0e4-46f9-bc85-d2d9ea5f9d13 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:25.748820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 666/1332] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:25.755260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666109-aae9-4d9b-8ec9-431828c77a38 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:25.755998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666109-aae9-4d9b-8ec9-431828c77a38 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:25.756193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7666109-aae9-4d9b-8ec9-431828c77a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:25.825534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:25.828314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d7666109-aae9-4d9b-8ec9-431828c77a38 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:25.830086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d7666109-aae9-4d9b-8ec9-431828c77a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:25.929241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 667/1333] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:26.206464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d129761-d0e4-46f9-bc85-d2d9ea5f9d13 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:26.208936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1d129761-d0e4-46f9-bc85-d2d9ea5f9d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:26.320292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d129761-d0e4-46f9-bc85-d2d9ea5f9d13 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:26.417840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c219a-0200-433e-93d2-b58bd898f82b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:26.418449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c219a-0200-433e-93d2-b58bd898f82b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:26.418720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c219a-0200-433e-93d2-b58bd898f82b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:26.418984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c219a-0200-433e-93d2-b58bd898f82b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:26.420579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7c9c219a-0200-433e-93d2-b58bd898f82b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:26.450130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:26.559050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c219a-0200-433e-93d2-b58bd898f82b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:26.560495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9c219a-0200-433e-93d2-b58bd898f82b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:26.579749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 667/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:28.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ec9e-ed79-44b4-bb55-3da6a90aea37 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:28.770771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ec9e-ed79-44b4-bb55-3da6a90aea37 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:28.770771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd10ec9e-ed79-44b4-bb55-3da6a90aea37 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:28.772019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 668/1335] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:28.778556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebce38-a623-4640-8c13-f6acc63ff456 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:28.779305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebce38-a623-4640-8c13-f6acc63ff456 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:28.779456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebce38-a623-4640-8c13-f6acc63ff456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:28.854568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-25ebce38-a623-4640-8c13-f6acc63ff456 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:28.855817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25ebce38-a623-4640-8c13-f6acc63ff456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:28.938387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:28.965505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 668/1336] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:29.243833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10ec9e-ed79-44b4-bb55-3da6a90aea37 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:29.246181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd10ec9e-ed79-44b4-bb55-3da6a90aea37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:29.357754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd10ec9e-ed79-44b4-bb55-3da6a90aea37 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:29.387653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:29.434135 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2727186f-9493-479d-87a9-257eea9c367f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:29.434778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727186f-9493-479d-87a9-257eea9c367f service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:29.435051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727186f-9493-479d-87a9-257eea9c367f service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:29.435317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2727186f-9493-479d-87a9-257eea9c367f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:29.436887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2727186f-9493-479d-87a9-257eea9c367f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:29.554567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727186f-9493-479d-87a9-257eea9c367f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:29.556159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2727186f-9493-479d-87a9-257eea9c367f service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:29.576063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 669/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:32.139022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09104cc2-c0a2-45d2-8414-4eb0f6fe68c2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:32.139877 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09104cc2-c0a2-45d2-8414-4eb0f6fe68c2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:32.139877 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09104cc2-c0a2-45d2-8414-4eb0f6fe68c2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:32.142007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 669/1338] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:32.148034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9f663-b42a-4a25-bb29-f3449ace3869 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:32.148756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9f663-b42a-4a25-bb29-f3449ace3869 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:32.148858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9f663-b42a-4a25-bb29-f3449ace3869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:32.225233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e6e9f663-b42a-4a25-bb29-f3449ace3869 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:32.226493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e6e9f663-b42a-4a25-bb29-f3449ace3869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:32.235203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:32.324815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 670/1339] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:32.608745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09104cc2-c0a2-45d2-8414-4eb0f6fe68c2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:32.610336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09104cc2-c0a2-45d2-8414-4eb0f6fe68c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:32.737852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09104cc2-c0a2-45d2-8414-4eb0f6fe68c2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:32.818400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de0b29-32e8-4d26-85b9-54f5edb3098a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:32.818912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de0b29-32e8-4d26-85b9-54f5edb3098a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:32.819210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de0b29-32e8-4d26-85b9-54f5edb3098a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:32.819473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de0b29-32e8-4d26-85b9-54f5edb3098a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:32.821072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-29de0b29-32e8-4d26-85b9-54f5edb3098a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:32.857887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:32.933961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de0b29-32e8-4d26-85b9-54f5edb3098a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:32.935461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de0b29-32e8-4d26-85b9-54f5edb3098a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:32.953672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 670/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:36.993668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9897d1-d4eb-4338-8d27-11599abf2cb9 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:36.994257 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9897d1-d4eb-4338-8d27-11599abf2cb9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:36.994257 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9897d1-d4eb-4338-8d27-11599abf2cb9 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:36.995537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 671/1341] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:37.000965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964a02-ca86-4066-a8d0-daa11635b0e9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:37.001429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964a02-ca86-4066-a8d0-daa11635b0e9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:37.001534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78964a02-ca86-4066-a8d0-daa11635b0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:37.068313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-78964a02-ca86-4066-a8d0-daa11635b0e9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:37.069411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-78964a02-ca86-4066-a8d0-daa11635b0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:37.158800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:37.170630 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 671/1342] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:37.448474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9897d1-d4eb-4338-8d27-11599abf2cb9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:37.450042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4c9897d1-d4eb-4338-8d27-11599abf2cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:37.562681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9897d1-d4eb-4338-8d27-11599abf2cb9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:37.603165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:37.640945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060fa05-e8d0-423b-add6-aefc994b685b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:37.641556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060fa05-e8d0-423b-add6-aefc994b685b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:37.641859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060fa05-e8d0-423b-add6-aefc994b685b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:37.642124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060fa05-e8d0-423b-add6-aefc994b685b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:37.643710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2060fa05-e8d0-423b-add6-aefc994b685b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:37.755716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060fa05-e8d0-423b-add6-aefc994b685b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:37.757067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060fa05-e8d0-423b-add6-aefc994b685b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:37.775546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 672/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:40.661797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379c8b7-ffe9-45b4-b5a5-6ab12c66b84e None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:40.662376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379c8b7-ffe9-45b4-b5a5-6ab12c66b84e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:40.662376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3379c8b7-ffe9-45b4-b5a5-6ab12c66b84e None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:40.663685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 672/1344] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:40.669029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf19eea-f651-4c6c-a078-da3909405dc1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:40.669479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf19eea-f651-4c6c-a078-da3909405dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:40.669583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf19eea-f651-4c6c-a078-da3909405dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:40.738748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-abf19eea-f651-4c6c-a078-da3909405dc1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:40.739970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-abf19eea-f651-4c6c-a078-da3909405dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:40.762481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:40.848546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 673/1345] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:41.133081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379c8b7-ffe9-45b4-b5a5-6ab12c66b84e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:41.134664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3379c8b7-ffe9-45b4-b5a5-6ab12c66b84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:41.245182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3379c8b7-ffe9-45b4-b5a5-6ab12c66b84e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:41.323447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4375bddc-1d79-43fd-88a0-763b5b184217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:41.324058 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375bddc-1d79-43fd-88a0-763b5b184217 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:41.324332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375bddc-1d79-43fd-88a0-763b5b184217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:41.324595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4375bddc-1d79-43fd-88a0-763b5b184217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:41.326194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4375bddc-1d79-43fd-88a0-763b5b184217 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:41.370759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:41.458222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375bddc-1d79-43fd-88a0-763b5b184217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:41.459661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4375bddc-1d79-43fd-88a0-763b5b184217 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:41.486542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 673/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:45.587410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffc226-2e1f-45a1-80ae-09b72c2934db None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:45.587916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffc226-2e1f-45a1-80ae-09b72c2934db None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:45.588472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffc226-2e1f-45a1-80ae-09b72c2934db None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:45.589935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 674/1347] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:03:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:45.596170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1a33-8815-4d8a-a6ab-5fbdf637b3ff None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:45.597795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1a33-8815-4d8a-a6ab-5fbdf637b3ff None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:45.597795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e1a33-8815-4d8a-a6ab-5fbdf637b3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:45.668920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-250e1a33-8815-4d8a-a6ab-5fbdf637b3ff None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:45.670221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-250e1a33-8815-4d8a-a6ab-5fbdf637b3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:45.771991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:45.774370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 674/1348] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:46.060182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffc226-2e1f-45a1-80ae-09b72c2934db None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:03:46.062744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3fffc226-2e1f-45a1-80ae-09b72c2934db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:46.173976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffc226-2e1f-45a1-80ae-09b72c2934db None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:03:46.223177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:46.251193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a89f34-130e-466e-8a84-7c38dbce99bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:03:46.251816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a89f34-130e-466e-8a84-7c38dbce99bf service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:46.252096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a89f34-130e-466e-8a84-7c38dbce99bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:46.252360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a89f34-130e-466e-8a84-7c38dbce99bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:46.254024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-64a89f34-130e-466e-8a84-7c38dbce99bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:46.378729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a89f34-130e-466e-8a84-7c38dbce99bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:03:46.380799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a89f34-130e-466e-8a84-7c38dbce99bf service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:03:46.407757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 675/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:03:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:03:51.379416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff2a47-05a7-4ad1-aa48-3d4dd0af8120 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:51.379984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff2a47-05a7-4ad1-aa48-3d4dd0af8120 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:51.379984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff2a47-05a7-4ad1-aa48-3d4dd0af8120 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:51.382632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 675/1350] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 14:03:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 14:03:51.390040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d324dd-454a-42f0-aad7-76f5dbaa55a6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:51.390471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d324dd-454a-42f0-aad7-76f5dbaa55a6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:51.390609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d324dd-454a-42f0-aad7-76f5dbaa55a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:51.488598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-43d324dd-454a-42f0-aad7-76f5dbaa55a6 None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:51.488598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-43d324dd-454a-42f0-aad7-76f5dbaa55a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:51.498475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:51.662056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 676/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 14:03:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:55.655093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2200486-1732-4e25-b488-7ab92eca7c02 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:55.655697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2200486-1732-4e25-b488-7ab92eca7c02 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:55.655697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2200486-1732-4e25-b488-7ab92eca7c02 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:55.657272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 676/1352] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 14:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:55.662560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2987b-0f19-4e70-8495-651b03b00adb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:55.663045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2987b-0f19-4e70-8495-651b03b00adb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:55.663262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d2987b-0f19-4e70-8495-651b03b00adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:55.741733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-07d2987b-0f19-4e70-8495-651b03b00adb None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:55.743032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-07d2987b-0f19-4e70-8495-651b03b00adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:55.813854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:03:55.851035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 677/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 14:03:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:03:57.338812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5836c9-37b9-4966-9d09-cb0e484c2e67 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:57.339372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5836c9-37b9-4966-9d09-cb0e484c2e67 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:57.339372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5836c9-37b9-4966-9d09-cb0e484c2e67 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:57.340907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 677/1354] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 14:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:03:57.345906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbd2e3-5cb2-4268-9c22-1e5bd5437805 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:03:57.346327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbd2e3-5cb2-4268-9c22-1e5bd5437805 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:03:57.346442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbd2e3-5cb2-4268-9c22-1e5bd5437805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:03:57.425289 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-25dbd2e3-5cb2-4268-9c22-1e5bd5437805 None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:03:57.426559 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-25dbd2e3-5cb2-4268-9c22-1e5bd5437805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:03:57.539086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 678/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 14:03:57 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:01.135869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdacb9-061d-412d-a046-6cd1ca9f0fd3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:01.136771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdacb9-061d-412d-a046-6cd1ca9f0fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:01.137488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bdacb9-061d-412d-a046-6cd1ca9f0fd3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:01.138951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 678/1356] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:01.144702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4031-3b25-45fb-8744-72979ae1b0c8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:01.145520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4031-3b25-45fb-8744-72979ae1b0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:01.145881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a4031-3b25-45fb-8744-72979ae1b0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:01.183656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:01.221018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ee9a4031-3b25-45fb-8744-72979ae1b0c8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:01.222265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ee9a4031-3b25-45fb-8744-72979ae1b0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:01.333055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 679/1357] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:01.623671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f4c93-aba1-4e4e-99d3-1a35df49b311 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:01.623949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f4c93-aba1-4e4e-99d3-1a35df49b311 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:01.624369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02f4c93-aba1-4e4e-99d3-1a35df49b311 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:01.626248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 679/1358] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 14:04:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:01.634060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6a20-0530-4290-813b-6245bd783b58 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:01.634811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6a20-0530-4290-813b-6245bd783b58 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:01.635916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f6a20-0530-4290-813b-6245bd783b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:01.721124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a87f6a20-0530-4290-813b-6245bd783b58 None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:01.722425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a87f6a20-0530-4290-813b-6245bd783b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:01.772625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:01.843132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 680/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 14:04:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:04:01.851069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f4c93-aba1-4e4e-99d3-1a35df49b311 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:01.852555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d02f4c93-aba1-4e4e-99d3-1a35df49b311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:02.045318 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02f4c93-aba1-4e4e-99d3-1a35df49b311 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:02.122527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:02.164760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a47cbd-beb1-44fc-be68-448372e4e855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:02.165432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a47cbd-beb1-44fc-be68-448372e4e855 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:02.165670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a47cbd-beb1-44fc-be68-448372e4e855 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:02.165929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a47cbd-beb1-44fc-be68-448372e4e855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:02.167562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-83a47cbd-beb1-44fc-be68-448372e4e855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:02.282695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a47cbd-beb1-44fc-be68-448372e4e855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:02.284490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a47cbd-beb1-44fc-be68-448372e4e855 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:02.311879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 680/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:04.720985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59a84-4f9e-4e5c-a981-24a8482b4e3c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:04.721471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59a84-4f9e-4e5c-a981-24a8482b4e3c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:04.721834 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce59a84-4f9e-4e5c-a981-24a8482b4e3c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:04.723768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 681/1361] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:04.729416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7251-de07-4e6a-9244-0d1dc706324f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:04.729949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7251-de07-4e6a-9244-0d1dc706324f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:04.730262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a7251-de07-4e6a-9244-0d1dc706324f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:04.798860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-450a7251-de07-4e6a-9244-0d1dc706324f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:04.800076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-450a7251-de07-4e6a-9244-0d1dc706324f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:04.841659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:04.913742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 681/1362] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:05.196950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce59a84-4f9e-4e5c-a981-24a8482b4e3c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:05.198712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1ce59a84-4f9e-4e5c-a981-24a8482b4e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:05.371554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce59a84-4f9e-4e5c-a981-24a8482b4e3c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:05.382382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:05.507372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1df9b8-4c5f-494d-a4fa-70c6346a4c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:05.507990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1df9b8-4c5f-494d-a4fa-70c6346a4c64 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:05.508263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1df9b8-4c5f-494d-a4fa-70c6346a4c64 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:05.508541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1df9b8-4c5f-494d-a4fa-70c6346a4c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:05.510296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed1df9b8-4c5f-494d-a4fa-70c6346a4c64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:05.599856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1df9b8-4c5f-494d-a4fa-70c6346a4c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:05.601524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1df9b8-4c5f-494d-a4fa-70c6346a4c64 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:05.620073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 682/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:08.062627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6394257-04e2-46e5-8dbe-a64f0f8819db None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:08.063505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6394257-04e2-46e5-8dbe-a64f0f8819db None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:08.063744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6394257-04e2-46e5-8dbe-a64f0f8819db None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:08.065386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 682/1364] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:08.070750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ffd71-4a15-4ba2-b124-1b482939397e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:08.071490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ffd71-4a15-4ba2-b124-1b482939397e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:08.071900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ffd71-4a15-4ba2-b124-1b482939397e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:08.087842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:08.141288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-011ffd71-4a15-4ba2-b124-1b482939397e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:08.142528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-011ffd71-4a15-4ba2-b124-1b482939397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:08.245556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 683/1365] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:08.546988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da3444-385d-44f8-b022-7cdad85c52ec None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:08.547238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da3444-385d-44f8-b022-7cdad85c52ec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:08.547504 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da3444-385d-44f8-b022-7cdad85c52ec None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:08.549034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 683/1366] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 14:04:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:08.555029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f10d5-b709-407a-b695-223cc2cb29e2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:08.555277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f10d5-b709-407a-b695-223cc2cb29e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:08.555575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37f10d5-b709-407a-b695-223cc2cb29e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:08.636413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e37f10d5-b709-407a-b695-223cc2cb29e2 None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:08.637628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e37f10d5-b709-407a-b695-223cc2cb29e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:08.674589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:08.767077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 684/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 14:04:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:08.776633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da3444-385d-44f8-b022-7cdad85c52ec None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:08.778746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-42da3444-385d-44f8-b022-7cdad85c52ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:08.933319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da3444-385d-44f8-b022-7cdad85c52ec None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:08.954682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:09.055208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5731b2-ee39-4de5-8c1b-2796874aa1c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:09.055963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5731b2-ee39-4de5-8c1b-2796874aa1c4 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:09.056228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5731b2-ee39-4de5-8c1b-2796874aa1c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:09.056499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5731b2-ee39-4de5-8c1b-2796874aa1c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:09.058313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6e5731b2-ee39-4de5-8c1b-2796874aa1c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:09.169858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5731b2-ee39-4de5-8c1b-2796874aa1c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:09.171558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5731b2-ee39-4de5-8c1b-2796874aa1c4 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:09.190908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 684/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:09.248855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc043e-370c-40a3-9420-4f8e1823e4bc None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:09.249424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc043e-370c-40a3-9420-4f8e1823e4bc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:09.249424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dc043e-370c-40a3-9420-4f8e1823e4bc None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:09.251052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 685/1369] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:09.256077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050a05-af9b-43cd-b28e-1f75cb8ac394 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:09.256380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050a05-af9b-43cd-b28e-1f75cb8ac394 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:09.256634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04050a05-af9b-43cd-b28e-1f75cb8ac394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:09.304585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:09.345297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-04050a05-af9b-43cd-b28e-1f75cb8ac394 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:09.346490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04050a05-af9b-43cd-b28e-1f75cb8ac394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:09.476534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 685/1370] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:04:09.802270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab805c-9f6b-4c75-9689-9a09bcea4c2f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:09.802791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab805c-9f6b-4c75-9689-9a09bcea4c2f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:09.803292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab805c-9f6b-4c75-9689-9a09bcea4c2f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:09.804979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 686/1371] 10.222.0.23 () {54 vars in 963 bytes} [Wed Oct 5 14:04:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:09.810482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d91c-9fb0-4c82-9016-7c86af5cef48 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:09.810825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d91c-9fb0-4c82-9016-7c86af5cef48 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:09.811157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c513d91c-9fb0-4c82-9016-7c86af5cef48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:09.886991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c513d91c-9fb0-4c82-9016-7c86af5cef48 None None] MFA Rules not processed for user `44fba9b1749b4172bc9479ca980fcf48`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:09.888322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c513d91c-9fb0-4c82-9016-7c86af5cef48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:09.939987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:09.978494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 686/1372] 10.222.0.23 () {58 vars in 1060 bytes} [Wed Oct 5 14:04:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:04:09.986352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab805c-9f6b-4c75-9689-9a09bcea4c2f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:09.988194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ebab805c-9f6b-4c75-9689-9a09bcea4c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:10.102305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab805c-9f6b-4c75-9689-9a09bcea4c2f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:10.147265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:10.192554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccb511a-647c-4c4d-934b-d8c093235f78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:10.194662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb511a-647c-4c4d-934b-d8c093235f78 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:10.194662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb511a-647c-4c4d-934b-d8c093235f78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:10.194662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccb511a-647c-4c4d-934b-d8c093235f78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:10.195916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fccb511a-647c-4c4d-934b-d8c093235f78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:10.312115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb511a-647c-4c4d-934b-d8c093235f78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:10.313948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fccb511a-647c-4c4d-934b-d8c093235f78 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:10.332064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 687/1373] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:11.435748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56397f9b-57c6-4030-b77f-172317c1ef1c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:11.436306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56397f9b-57c6-4030-b77f-172317c1ef1c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:11.436306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56397f9b-57c6-4030-b77f-172317c1ef1c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:11.437668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 687/1374] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:11.443331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d584-dfdc-487d-8ed0-f3234b7803f2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:11.443742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d584-dfdc-487d-8ed0-f3234b7803f2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:11.444021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97d584-dfdc-487d-8ed0-f3234b7803f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:11.514271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5a97d584-dfdc-487d-8ed0-f3234b7803f2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:11.515537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5a97d584-dfdc-487d-8ed0-f3234b7803f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:11.536928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:11.618341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 688/1375] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:11.918844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56397f9b-57c6-4030-b77f-172317c1ef1c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:11.920576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-56397f9b-57c6-4030-b77f-172317c1ef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:12.085102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56397f9b-57c6-4030-b77f-172317c1ef1c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:12.139533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:12.216507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9240ef-1893-4614-8179-68a9507d4521 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:12.217129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9240ef-1893-4614-8179-68a9507d4521 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:12.217401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9240ef-1893-4614-8179-68a9507d4521 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:12.217690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9240ef-1893-4614-8179-68a9507d4521 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:12.219283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4b9240ef-1893-4614-8179-68a9507d4521 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:12.314138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9240ef-1893-4614-8179-68a9507d4521 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:12.315784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9240ef-1893-4614-8179-68a9507d4521 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:12.334517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 688/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:14.538185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a4362-f12e-49a3-9f30-10898aaff3b0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:14.538754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a4362-f12e-49a3-9f30-10898aaff3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:14.538964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019a4362-f12e-49a3-9f30-10898aaff3b0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:14.540470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 689/1377] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:14.545983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb0749-1b6b-4249-adeb-0df29bccade4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:14.546443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb0749-1b6b-4249-adeb-0df29bccade4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:14.546534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb0749-1b6b-4249-adeb-0df29bccade4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:14.622514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-61fb0749-1b6b-4249-adeb-0df29bccade4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:14.623815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-61fb0749-1b6b-4249-adeb-0df29bccade4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:14.633157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:14.733691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 689/1378] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:15.016488 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a4362-f12e-49a3-9f30-10898aaff3b0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:15.018053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-019a4362-f12e-49a3-9f30-10898aaff3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:15.166808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019a4362-f12e-49a3-9f30-10898aaff3b0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:15.171689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:15.286594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36a570a-60bd-4888-abfd-800a5165a9ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:15.287139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a570a-60bd-4888-abfd-800a5165a9ee service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:15.287423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a570a-60bd-4888-abfd-800a5165a9ee service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:15.287657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36a570a-60bd-4888-abfd-800a5165a9ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:15.289274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f36a570a-60bd-4888-abfd-800a5165a9ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:15.381953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a570a-60bd-4888-abfd-800a5165a9ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:15.383588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36a570a-60bd-4888-abfd-800a5165a9ee service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:15.404362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 690/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:17.653369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068508c-0719-4795-a7cd-a97ce4573101 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:17.653894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068508c-0719-4795-a7cd-a97ce4573101 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:17.654092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068508c-0719-4795-a7cd-a97ce4573101 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:17.655508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 690/1380] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:17.662676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5bc8-f6ad-412a-b16c-131d351cefa2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:17.663161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5bc8-f6ad-412a-b16c-131d351cefa2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:17.663259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827b5bc8-f6ad-412a-b16c-131d351cefa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:17.676727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:17.765152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-827b5bc8-f6ad-412a-b16c-131d351cefa2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:17.766975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-827b5bc8-f6ad-412a-b16c-131d351cefa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:17.914105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 691/1381] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:18.216720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068508c-0719-4795-a7cd-a97ce4573101 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:18.218717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5068508c-0719-4795-a7cd-a97ce4573101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:18.373345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:18.384467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068508c-0719-4795-a7cd-a97ce4573101 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:18.513754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fea64f-e4be-4ae6-852a-6a47a0d95eae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:18.514699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea64f-e4be-4ae6-852a-6a47a0d95eae service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:18.514986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea64f-e4be-4ae6-852a-6a47a0d95eae service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:18.515278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fea64f-e4be-4ae6-852a-6a47a0d95eae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:18.517805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c2fea64f-e4be-4ae6-852a-6a47a0d95eae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:18.614548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fea64f-e4be-4ae6-852a-6a47a0d95eae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:18.616159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fea64f-e4be-4ae6-852a-6a47a0d95eae service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:18.627668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:18.637184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 691/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:20.924193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229645-eb8b-4550-aa83-4bb0c5ca5cc2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:20.924836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229645-eb8b-4550-aa83-4bb0c5ca5cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:20.924836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f229645-eb8b-4550-aa83-4bb0c5ca5cc2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:20.926254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 692/1383] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:20.932229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949bf54-3bc1-4086-9813-7c4f326d8c3e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:20.932759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949bf54-3bc1-4086-9813-7c4f326d8c3e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:20.932862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949bf54-3bc1-4086-9813-7c4f326d8c3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:21.004524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8949bf54-3bc1-4086-9813-7c4f326d8c3e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:21.006275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8949bf54-3bc1-4086-9813-7c4f326d8c3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:21.120040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 692/1384] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:21.459214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f229645-eb8b-4550-aa83-4bb0c5ca5cc2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:21.460928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1f229645-eb8b-4550-aa83-4bb0c5ca5cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:21.551253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:21.613836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f229645-eb8b-4550-aa83-4bb0c5ca5cc2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:21.736133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4fc40-17d5-401c-a53f-7dca5ed8bd96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:21.736667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4fc40-17d5-401c-a53f-7dca5ed8bd96 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:21.736969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4fc40-17d5-401c-a53f-7dca5ed8bd96 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:21.737254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4fc40-17d5-401c-a53f-7dca5ed8bd96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:21.739004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ac4fc40-17d5-401c-a53f-7dca5ed8bd96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:21.796036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:21.837307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4fc40-17d5-401c-a53f-7dca5ed8bd96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:21.839039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4fc40-17d5-401c-a53f-7dca5ed8bd96 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:21.857873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 693/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:24.381681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebb5b-c6a2-4db2-8e07-e672fa892b6d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:24.382349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebb5b-c6a2-4db2-8e07-e672fa892b6d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:24.382349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadebb5b-c6a2-4db2-8e07-e672fa892b6d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:24.384075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 693/1386] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:24.390553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f41c1-1286-4df2-9a05-294a2e70720a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:24.390999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f41c1-1286-4df2-9a05-294a2e70720a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:24.391113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f41c1-1286-4df2-9a05-294a2e70720a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:24.463382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ee4f41c1-1286-4df2-9a05-294a2e70720a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:24.464715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ee4f41c1-1286-4df2-9a05-294a2e70720a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:24.576793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 694/1387] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:24.876047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebb5b-c6a2-4db2-8e07-e672fa892b6d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:24.877682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fadebb5b-c6a2-4db2-8e07-e672fa892b6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:24.973563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:25.056644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadebb5b-c6a2-4db2-8e07-e672fa892b6d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:25.192641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b7d630-f524-44ea-8733-2af493db35cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:25.193274 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d630-f524-44ea-8733-2af493db35cc service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:25.193570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d630-f524-44ea-8733-2af493db35cc service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:25.193894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b7d630-f524-44ea-8733-2af493db35cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:25.195568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09b7d630-f524-44ea-8733-2af493db35cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:25.237621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:25.291041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7d630-f524-44ea-8733-2af493db35cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:25.292747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b7d630-f524-44ea-8733-2af493db35cc service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:25.310399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 694/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:27.488222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb48a1-4e4a-4ae8-861e-428630ecf96f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:27.488801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb48a1-4e4a-4ae8-861e-428630ecf96f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:27.488801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb48a1-4e4a-4ae8-861e-428630ecf96f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:27.490381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 695/1389] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:27.495764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c74f-bf37-447a-b0b6-41552f13f34a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:27.496183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c74f-bf37-447a-b0b6-41552f13f34a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:27.496271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7532c74f-bf37-447a-b0b6-41552f13f34a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:27.565100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7532c74f-bf37-447a-b0b6-41552f13f34a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:27.566401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7532c74f-bf37-447a-b0b6-41552f13f34a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:27.666451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 695/1390] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:27.952547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb48a1-4e4a-4ae8-861e-428630ecf96f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:27.954099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-23bb48a1-4e4a-4ae8-861e-428630ecf96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:27.988513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:28.103829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb48a1-4e4a-4ae8-861e-428630ecf96f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:28.210916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:28.222866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b7c0f3-57e7-4b39-8e97-6c3f6e63ffe6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:28.223492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c0f3-57e7-4b39-8e97-6c3f6e63ffe6 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:28.223743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c0f3-57e7-4b39-8e97-6c3f6e63ffe6 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:28.224007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b7c0f3-57e7-4b39-8e97-6c3f6e63ffe6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:28.225581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c2b7c0f3-57e7-4b39-8e97-6c3f6e63ffe6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:28.293410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977b67-208d-4a82-bba2-3d71be636fad None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:28.293696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977b67-208d-4a82-bba2-3d71be636fad None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:28.293984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af977b67-208d-4a82-bba2-3d71be636fad None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:28.295291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 696/1391] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 14:04:28.300233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856549-b41d-4fbc-9763-48ece78511fb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:28.300510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856549-b41d-4fbc-9763-48ece78511fb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:28.300778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b856549-b41d-4fbc-9763-48ece78511fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:28.316192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:28.322657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7c0f3-57e7-4b39-8e97-6c3f6e63ffe6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:28.324425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b7c0f3-57e7-4b39-8e97-6c3f6e63ffe6 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:28.342985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 696/1392] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:28.373655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0b856549-b41d-4fbc-9763-48ece78511fb None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:28.374942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b856549-b41d-4fbc-9763-48ece78511fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:28.481151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 697/1393] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:04:28.777321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983416a-308b-459c-8f22-8c561f9d4db7 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:28.777836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983416a-308b-459c-8f22-8c561f9d4db7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:28.777932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7983416a-308b-459c-8f22-8c561f9d4db7 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:28.779262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 697/1394] 10.222.0.23 () {54 vars in 962 bytes} [Wed Oct 5 14:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:28.785385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9438-e936-4b9b-abc0-514217975a6d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:28.785876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9438-e936-4b9b-abc0-514217975a6d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:28.785971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd9438-e936-4b9b-abc0-514217975a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:28.873836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3dbd9438-e936-4b9b-abc0-514217975a6d None None] MFA Rules not processed for user `417b661f770846f189c36d0804b6dce5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:28.875075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3dbd9438-e936-4b9b-abc0-514217975a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:28.911451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:28.954183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 698/1395] 10.222.0.23 () {58 vars in 1059 bytes} [Wed Oct 5 14:04:28 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:28.963497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983416a-308b-459c-8f22-8c561f9d4db7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:28.965051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7983416a-308b-459c-8f22-8c561f9d4db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:29.068128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7983416a-308b-459c-8f22-8c561f9d4db7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:29.083593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:29.148161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a5deae-e881-41f2-89a1-1073d0c11301 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417b661f770846f189c36d0804b6dce5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:29.148771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5deae-e881-41f2-89a1-1073d0c11301 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:29.149050 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5deae-e881-41f2-89a1-1073d0c11301 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:29.149321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a5deae-e881-41f2-89a1-1073d0c11301 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:29.150930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e9a5deae-e881-41f2-89a1-1073d0c11301 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:29.265956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5deae-e881-41f2-89a1-1073d0c11301 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:29.267441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a5deae-e881-41f2-89a1-1073d0c11301 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:29.288357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 698/1396] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:30.023399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6c592-3c15-42b6-8830-3969488c8626 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:30.023897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6c592-3c15-42b6-8830-3969488c8626 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:30.023996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6c592-3c15-42b6-8830-3969488c8626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:30.093368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-01a6c592-3c15-42b6-8830-3969488c8626 None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:30.094733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01a6c592-3c15-42b6-8830-3969488c8626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:30.186812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 699/1397] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:04:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:30.233814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6826-5a24-4ebb-a417-604da3ab0121 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:30.234357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6826-5a24-4ebb-a417-604da3ab0121 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:30.234357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff6826-5a24-4ebb-a417-604da3ab0121 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:30.235955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 699/1398] 10.222.0.23 () {54 vars in 943 bytes} [Wed Oct 5 14:04:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:30.242570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641ac2-e024-42bb-a163-5c8e49c3e778 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:30.242839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641ac2-e024-42bb-a163-5c8e49c3e778 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:30.243104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641ac2-e024-42bb-a163-5c8e49c3e778 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:30.304050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:30.323619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ae641ac2-e024-42bb-a163-5c8e49c3e778 None None] MFA Rules not processed for user `01286f6f65b44519b645927f6c70ec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:30.324827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ae641ac2-e024-42bb-a163-5c8e49c3e778 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:30.432517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 700/1399] 10.222.0.23 () {58 vars in 1040 bytes} [Wed Oct 5 14:04:30 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:30.442680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff6826-5a24-4ebb-a417-604da3ab0121 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:30.444436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6cff6826-5a24-4ebb-a417-604da3ab0121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:30.490631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:30.539992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362e717-341e-4ed2-a8da-ac4483342bb3 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:30.540542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362e717-341e-4ed2-a8da-ac4483342bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:30.540826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3362e717-341e-4ed2-a8da-ac4483342bb3 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:30.542477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 701/1400] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:30.547559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382edf-3d11-46d2-aca0-c4f9c41dd36d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:30.547841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382edf-3d11-46d2-aca0-c4f9c41dd36d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:30.548134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382edf-3d11-46d2-aca0-c4f9c41dd36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:30.584930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff6826-5a24-4ebb-a417-604da3ab0121 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:30.617854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-52382edf-3d11-46d2-aca0-c4f9c41dd36d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:30.619056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-52382edf-3d11-46d2-aca0-c4f9c41dd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:30.676678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:30.689569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ce52-c6cf-4baf-b295-95179b53e11b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:30.690194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ce52-c6cf-4baf-b295-95179b53e11b service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:30.690484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ce52-c6cf-4baf-b295-95179b53e11b service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:30.690729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ce52-c6cf-4baf-b295-95179b53e11b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:30.692376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d00ce52-c6cf-4baf-b295-95179b53e11b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:30.721171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 702/1401] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:30.725365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:30.778391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00ce52-c6cf-4baf-b295-95179b53e11b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:30.780027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d00ce52-c6cf-4baf-b295-95179b53e11b service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:30.788583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 700/1402] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:04:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:04:31.005172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52382edf-3d11-46d2-aca0-c4f9c41dd36d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:31.008341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-52382edf-3d11-46d2-aca0-c4f9c41dd36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:31.130400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52382edf-3d11-46d2-aca0-c4f9c41dd36d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:31.210022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:31.256663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1bf56-d1a0-4896-bb33-79db66d1a4a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:31.259156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1bf56-d1a0-4896-bb33-79db66d1a4a2 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:31.259436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1bf56-d1a0-4896-bb33-79db66d1a4a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:31.259714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1bf56-d1a0-4896-bb33-79db66d1a4a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:31.261462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8de1bf56-d1a0-4896-bb33-79db66d1a4a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:31.355792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1bf56-d1a0-4896-bb33-79db66d1a4a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:31.357871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1bf56-d1a0-4896-bb33-79db66d1a4a2 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:31.376280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 703/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:33.613728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6811c6-edb7-45f6-9442-5ad6cb8655ac None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:33.624510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6811c6-edb7-45f6-9442-5ad6cb8655ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:33.624510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6811c6-edb7-45f6-9442-5ad6cb8655ac None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:33.624510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 701/1404] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:33.628194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e106c-37d0-4cc5-b872-63d383f2bf2e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:33.628834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e106c-37d0-4cc5-b872-63d383f2bf2e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:33.631179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42e106c-37d0-4cc5-b872-63d383f2bf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:33.740674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b42e106c-37d0-4cc5-b872-63d383f2bf2e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:33.742057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b42e106c-37d0-4cc5-b872-63d383f2bf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:33.776612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:33.848105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 704/1405] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:34.153226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6811c6-edb7-45f6-9442-5ad6cb8655ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:34.154797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6c6811c6-edb7-45f6-9442-5ad6cb8655ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:34.320263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6811c6-edb7-45f6-9442-5ad6cb8655ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:34.346234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:34.451177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c675ab-a82e-4be4-8863-7912c491cb76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:34.451788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c675ab-a82e-4be4-8863-7912c491cb76 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:34.452068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c675ab-a82e-4be4-8863-7912c491cb76 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:34.452336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c675ab-a82e-4be4-8863-7912c491cb76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:34.453987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e0c675ab-a82e-4be4-8863-7912c491cb76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:34.558794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c675ab-a82e-4be4-8863-7912c491cb76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:34.560421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c675ab-a82e-4be4-8863-7912c491cb76 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:34.580629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 702/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:36.778196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c86a89-ffa5-409f-88ed-2fe5f04b497a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:36.778768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c86a89-ffa5-409f-88ed-2fe5f04b497a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:36.778768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c86a89-ffa5-409f-88ed-2fe5f04b497a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:36.779996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 705/1407] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:04:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:36.785339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d975a9f-b20c-4c9f-ab55-b5a3cb782f11 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:36.785824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d975a9f-b20c-4c9f-ab55-b5a3cb782f11 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:36.785912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d975a9f-b20c-4c9f-ab55-b5a3cb782f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:36.826803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:36.860674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9d975a9f-b20c-4c9f-ab55-b5a3cb782f11 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:36.862394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9d975a9f-b20c-4c9f-ab55-b5a3cb782f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:36.966510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 703/1408] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:37.253891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c86a89-ffa5-409f-88ed-2fe5f04b497a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:37.255671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-74c86a89-ffa5-409f-88ed-2fe5f04b497a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:37.416696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c86a89-ffa5-409f-88ed-2fe5f04b497a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:37.434121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:37.544510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c814bd3a-c8fe-425a-b130-ec1454e8632f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:37.545119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814bd3a-c8fe-425a-b130-ec1454e8632f service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:37.545395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814bd3a-c8fe-425a-b130-ec1454e8632f service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:37.545678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c814bd3a-c8fe-425a-b130-ec1454e8632f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:37.547286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c814bd3a-c8fe-425a-b130-ec1454e8632f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:37.644759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814bd3a-c8fe-425a-b130-ec1454e8632f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:37.646504 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c814bd3a-c8fe-425a-b130-ec1454e8632f service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:37.665514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 706/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:46.964794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=53 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:47.663239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:53.703521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86af5d-d33f-4696-91ca-9103ac883036 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:53.704138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86af5d-d33f-4696-91ca-9103ac883036 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:53.704138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86af5d-d33f-4696-91ca-9103ac883036 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:53.705572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 704/1410] 10.222.0.23 () {54 vars in 918 bytes} [Wed Oct 5 14:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:53.711528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865842-d692-47e2-80ea-bd344dcae8fb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:53.711969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865842-d692-47e2-80ea-bd344dcae8fb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:53.712074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e865842-d692-47e2-80ea-bd344dcae8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:53.790768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2e865842-d692-47e2-80ea-bd344dcae8fb None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:53.791882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2e865842-d692-47e2-80ea-bd344dcae8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:53.902068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 707/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Oct 5 14:04:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:53.917242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2256f6-3677-4614-b6f6-9a8a74e4e8e2 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:53.917515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2256f6-3677-4614-b6f6-9a8a74e4e8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:53.917813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2256f6-3677-4614-b6f6-9a8a74e4e8e2 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:53.919129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 705/1412] 10.222.0.23 () {54 vars in 918 bytes} [Wed Oct 5 14:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:53.923855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb59c4-1da8-466e-9376-7fb764c2b63e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:53.924139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb59c4-1da8-466e-9376-7fb764c2b63e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:53.924408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb59c4-1da8-466e-9376-7fb764c2b63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:53.999710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:54.003267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5bfb59c4-1da8-466e-9376-7fb764c2b63e None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:54.005206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5bfb59c4-1da8-466e-9376-7fb764c2b63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:54.125098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 708/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Oct 5 14:04:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:56.695221 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d492f5-aa68-498b-8b3c-a7b9a948d9af None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:56.696131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d492f5-aa68-498b-8b3c-a7b9a948d9af None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:56.696131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d492f5-aa68-498b-8b3c-a7b9a948d9af None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:56.698351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:04:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:56.704850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f864c8e-1c09-47bd-839b-6c6d13d661d7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:56.705613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f864c8e-1c09-47bd-839b-6c6d13d661d7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:56.705810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f864c8e-1c09-47bd-839b-6c6d13d661d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:56.787661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8f864c8e-1c09-47bd-839b-6c6d13d661d7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:56.788783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8f864c8e-1c09-47bd-839b-6c6d13d661d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:56.868427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:56.912991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:57.209793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d492f5-aa68-498b-8b3c-a7b9a948d9af None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:57.211941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-28d492f5-aa68-498b-8b3c-a7b9a948d9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:57.366890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d492f5-aa68-498b-8b3c-a7b9a948d9af None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:57.451480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:57.495295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182491cc-171b-4ff9-8b86-071bf93306d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:57.495906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182491cc-171b-4ff9-8b86-071bf93306d0 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:57.496179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182491cc-171b-4ff9-8b86-071bf93306d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:57.496447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182491cc-171b-4ff9-8b86-071bf93306d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:57.498121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-182491cc-171b-4ff9-8b86-071bf93306d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:57.595857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182491cc-171b-4ff9-8b86-071bf93306d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:57.597437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182491cc-171b-4ff9-8b86-071bf93306d0 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:57.616368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 707/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:04:58.745395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ea060-3410-4cd7-b42c-440804cad560 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:58.746076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ea060-3410-4cd7-b42c-440804cad560 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:58.746076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ea060-3410-4cd7-b42c-440804cad560 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:58.747261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:04:58.754120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96f578-66c5-47ac-8bdc-eb3728a037a4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:58.754579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96f578-66c5-47ac-8bdc-eb3728a037a4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:58.754689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96f578-66c5-47ac-8bdc-eb3728a037a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:58.842960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9f96f578-66c5-47ac-8bdc-eb3728a037a4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:04:58.844090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f96f578-66c5-47ac-8bdc-eb3728a037a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:58.876479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:58.950567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:04:59.118002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ea060-3410-4cd7-b42c-440804cad560 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:04:59.120372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-612ea060-3410-4cd7-b42c-440804cad560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:59.277905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ea060-3410-4cd7-b42c-440804cad560 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:04:59.321067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:04:59.395377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad357a9-fdb3-4f77-8c1a-66d02e611b54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:04:59.395900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad357a9-fdb3-4f77-8c1a-66d02e611b54 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:04:59.396172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad357a9-fdb3-4f77-8c1a-66d02e611b54 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:04:59.396439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad357a9-fdb3-4f77-8c1a-66d02e611b54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:04:59.398087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0ad357a9-fdb3-4f77-8c1a-66d02e611b54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:04:59.493362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad357a9-fdb3-4f77-8c1a-66d02e611b54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:04:59.495086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad357a9-fdb3-4f77-8c1a-66d02e611b54 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:04:59.517069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 711/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:04:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:00.643715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1853e9-8320-4ed1-87b7-4db2132e98cb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:00.644582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1853e9-8320-4ed1-87b7-4db2132e98cb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:00.644582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1853e9-8320-4ed1-87b7-4db2132e98cb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:00.646702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:05:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:00.657285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9a989-708c-44a7-8fb9-f9cb567dd2f7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:00.657782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9a989-708c-44a7-8fb9-f9cb567dd2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:00.657885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a9a989-708c-44a7-8fb9-f9cb567dd2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:00.719031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:00.743759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-07a9a989-708c-44a7-8fb9-f9cb567dd2f7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:00.744983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-07a9a989-708c-44a7-8fb9-f9cb567dd2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:00.854563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:01.012445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1853e9-8320-4ed1-87b7-4db2132e98cb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:01.014917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9a1853e9-8320-4ed1-87b7-4db2132e98cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:01.172942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1853e9-8320-4ed1-87b7-4db2132e98cb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:01.185685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:01.292959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59825f3-77b8-4d0d-bf83-fa2b179e64c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:01.293498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59825f3-77b8-4d0d-bf83-fa2b179e64c1 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:01.293822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59825f3-77b8-4d0d-bf83-fa2b179e64c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:01.294098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59825f3-77b8-4d0d-bf83-fa2b179e64c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:01.295750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b59825f3-77b8-4d0d-bf83-fa2b179e64c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:01.392353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59825f3-77b8-4d0d-bf83-fa2b179e64c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:01.394003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59825f3-77b8-4d0d-bf83-fa2b179e64c1 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:01.413203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 710/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:02.507233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a816b-7ee1-4573-b65f-408dd0a04ce8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:02.507832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a816b-7ee1-4573-b65f-408dd0a04ce8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:02.507832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a816b-7ee1-4573-b65f-408dd0a04ce8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:02.509316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:02.519815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c495d-cbb5-4ab4-9959-7164bc5e1741 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:02.520569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c495d-cbb5-4ab4-9959-7164bc5e1741 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:02.520726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c495d-cbb5-4ab4-9959-7164bc5e1741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:02.544504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:02.605400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ab5c495d-cbb5-4ab4-9959-7164bc5e1741 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:02.606597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab5c495d-cbb5-4ab4-9959-7164bc5e1741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:02.834253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:02.988528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a816b-7ee1-4573-b65f-408dd0a04ce8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:02.990118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ed2a816b-7ee1-4573-b65f-408dd0a04ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:03.108870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:03.149512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2a816b-7ee1-4573-b65f-408dd0a04ce8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:03.270599 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c5b6ec-8b02-4afd-8bb0-ed843efdc08b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:03.271213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b6ec-8b02-4afd-8bb0-ed843efdc08b service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:03.271483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b6ec-8b02-4afd-8bb0-ed843efdc08b service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:03.271748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c5b6ec-8b02-4afd-8bb0-ed843efdc08b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:03.273341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-18c5b6ec-8b02-4afd-8bb0-ed843efdc08b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:03.358469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:03.373231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5b6ec-8b02-4afd-8bb0-ed843efdc08b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:03.374902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c5b6ec-8b02-4afd-8bb0-ed843efdc08b service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:03.392830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 714/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:04.504630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d602d-1c63-45bb-ba0e-fb93dbe20822 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:04.505441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d602d-1c63-45bb-ba0e-fb93dbe20822 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:04.505441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178d602d-1c63-45bb-ba0e-fb93dbe20822 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:04.507432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:05:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:04.514249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02763f-6209-4dd5-bdfb-87a841d8081f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:04.514810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02763f-6209-4dd5-bdfb-87a841d8081f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:04.514810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c02763f-6209-4dd5-bdfb-87a841d8081f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:04.593673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1c02763f-6209-4dd5-bdfb-87a841d8081f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:04.594892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1c02763f-6209-4dd5-bdfb-87a841d8081f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:04.715314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:05:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:04.867483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d602d-1c63-45bb-ba0e-fb93dbe20822 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:04.869729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-178d602d-1c63-45bb-ba0e-fb93dbe20822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:04.961752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:05.033465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178d602d-1c63-45bb-ba0e-fb93dbe20822 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:05.162321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1366f-1bb6-4537-afb3-8582a50ccd18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:05.162931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1366f-1bb6-4537-afb3-8582a50ccd18 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:05.163202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1366f-1bb6-4537-afb3-8582a50ccd18 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:05.163463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1366f-1bb6-4537-afb3-8582a50ccd18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:05.165043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8f1366f-1bb6-4537-afb3-8582a50ccd18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:05.215855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:05.258007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1366f-1bb6-4537-afb3-8582a50ccd18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:05.259623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f1366f-1bb6-4537-afb3-8582a50ccd18 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:05.279179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 713/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:06.333500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32377f-3514-4001-825f-274fe3a07ac1 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:06.334438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32377f-3514-4001-825f-274fe3a07ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:06.334438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32377f-3514-4001-825f-274fe3a07ac1 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:06.336659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:05:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:06.343245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747bf9-228e-420a-bde4-4b437cb7c847 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:06.343727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747bf9-228e-420a-bde4-4b437cb7c847 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:06.343727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d747bf9-228e-420a-bde4-4b437cb7c847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:06.433320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7d747bf9-228e-420a-bde4-4b437cb7c847 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:06.435196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d747bf9-228e-420a-bde4-4b437cb7c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:06.540104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:06.700346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32377f-3514-4001-825f-274fe3a07ac1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:06.703039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5d32377f-3514-4001-825f-274fe3a07ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:06.766459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:06.867177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d32377f-3514-4001-825f-274fe3a07ac1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:06.994435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8fc68-ca6a-4c93-a2a2-82bd8935bdcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:06.994958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8fc68-ca6a-4c93-a2a2-82bd8935bdcc service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:06.995235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8fc68-ca6a-4c93-a2a2-82bd8935bdcc service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:06.995499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8fc68-ca6a-4c93-a2a2-82bd8935bdcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:06.997321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09a8fc68-ca6a-4c93-a2a2-82bd8935bdcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:07.004684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:07.102299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8fc68-ca6a-4c93-a2a2-82bd8935bdcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:07.103913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8fc68-ca6a-4c93-a2a2-82bd8935bdcc service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:07.122247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 717/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:08.173548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859afb6-1735-47a7-9da0-4804126226b8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:08.174278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859afb6-1735-47a7-9da0-4804126226b8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:08.174278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859afb6-1735-47a7-9da0-4804126226b8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:08.175468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:08.182996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a71b64-f505-4976-8f84-999f273af495 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:08.183721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a71b64-f505-4976-8f84-999f273af495 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:08.183910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a71b64-f505-4976-8f84-999f273af495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:08.270654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e9a71b64-f505-4976-8f84-999f273af495 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:08.271873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e9a71b64-f505-4976-8f84-999f273af495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:08.344893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:08.375724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:08.530387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859afb6-1735-47a7-9da0-4804126226b8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:08.532002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b859afb6-1735-47a7-9da0-4804126226b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:08.549996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:08.682523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859afb6-1735-47a7-9da0-4804126226b8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:08.781369 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:08.809586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae92127-3dc5-4b3a-aea6-67abef4d32e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:08.810235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92127-3dc5-4b3a-aea6-67abef4d32e3 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:08.810512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92127-3dc5-4b3a-aea6-67abef4d32e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:08.810775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae92127-3dc5-4b3a-aea6-67abef4d32e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:08.812416 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5ae92127-3dc5-4b3a-aea6-67abef4d32e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:08.912735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae92127-3dc5-4b3a-aea6-67abef4d32e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:08.914388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae92127-3dc5-4b3a-aea6-67abef4d32e3 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:08.934360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 716/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:09.989230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85140691-9c52-4312-b99c-598583529397 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:09.989904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85140691-9c52-4312-b99c-598583529397 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:09.989904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85140691-9c52-4312-b99c-598583529397 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:09.991181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:09.998630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599acd-4504-48ab-b129-b71f3c6e715a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:09.999183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599acd-4504-48ab-b129-b71f3c6e715a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:09.999296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599acd-4504-48ab-b129-b71f3c6e715a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:10.077459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ee599acd-4504-48ab-b129-b71f3c6e715a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:10.078747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ee599acd-4504-48ab-b129-b71f3c6e715a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:10.123660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:10.183252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:10.333007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85140691-9c52-4312-b99c-598583529397 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:10.335295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-85140691-9c52-4312-b99c-598583529397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:10.498820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85140691-9c52-4312-b99c-598583529397 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:10.559035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:10.623009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a957b4c-245c-4cc3-b80c-24c36916e160 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:10.623560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a957b4c-245c-4cc3-b80c-24c36916e160 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:10.623833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a957b4c-245c-4cc3-b80c-24c36916e160 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:10.624105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a957b4c-245c-4cc3-b80c-24c36916e160 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:10.625753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8a957b4c-245c-4cc3-b80c-24c36916e160 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:10.719703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a957b4c-245c-4cc3-b80c-24c36916e160 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:10.721410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a957b4c-245c-4cc3-b80c-24c36916e160 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:10.746100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 720/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:11.795592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e68472-0b92-440f-97ec-6cc8291c919f None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:11.796376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e68472-0b92-440f-97ec-6cc8291c919f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:11.796376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e68472-0b92-440f-97ec-6cc8291c919f None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:11.797742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:05:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:11.805048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f09c-4579-4975-9ab4-2c61c8e3aa62 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:11.805529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f09c-4579-4975-9ab4-2c61c8e3aa62 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:11.805629 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0756f09c-4579-4975-9ab4-2c61c8e3aa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:11.892570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0756f09c-4579-4975-9ab4-2c61c8e3aa62 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:11.893858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0756f09c-4579-4975-9ab4-2c61c8e3aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:11.895979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:12.007090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:12.157283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e68472-0b92-440f-97ec-6cc8291c919f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:12.159409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-75e68472-0b92-440f-97ec-6cc8291c919f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:12.322691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e68472-0b92-440f-97ec-6cc8291c919f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:12.350562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:12.453533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b90666-142a-4462-8a47-a569ab5a99ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:12.454096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90666-142a-4462-8a47-a569ab5a99ac service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:12.454374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90666-142a-4462-8a47-a569ab5a99ac service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:12.454645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b90666-142a-4462-8a47-a569ab5a99ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:12.456249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-62b90666-142a-4462-8a47-a569ab5a99ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:12.552958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b90666-142a-4462-8a47-a569ab5a99ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:12.554591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b90666-142a-4462-8a47-a569ab5a99ac service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:12.573187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 719/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:14.618411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0d5c4-44e1-4b17-a35c-8fd4c08bac3a None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:14.619090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0d5c4-44e1-4b17-a35c-8fd4c08bac3a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:14.619090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0d5c4-44e1-4b17-a35c-8fd4c08bac3a None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:14.620311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:14.626965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3d2ed-8658-42e2-b824-9dc278dd764e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:14.627441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3d2ed-8658-42e2-b824-9dc278dd764e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:14.627441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3d2ed-8658-42e2-b824-9dc278dd764e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:14.679104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:14.706294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c7d3d2ed-8658-42e2-b824-9dc278dd764e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:14.707506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7d3d2ed-8658-42e2-b824-9dc278dd764e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:14.816982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:15.104811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0d5c4-44e1-4b17-a35c-8fd4c08bac3a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:15.107355 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0ca0d5c4-44e1-4b17-a35c-8fd4c08bac3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:15.216542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0d5c4-44e1-4b17-a35c-8fd4c08bac3a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:15.251240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:15.295083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15433c9-fef3-4f1e-9df9-1bb2bcfa8c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:15.295726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15433c9-fef3-4f1e-9df9-1bb2bcfa8c39 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:15.296050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15433c9-fef3-4f1e-9df9-1bb2bcfa8c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:15.296325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15433c9-fef3-4f1e-9df9-1bb2bcfa8c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:15.297980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f15433c9-fef3-4f1e-9df9-1bb2bcfa8c39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:15.414113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15433c9-fef3-4f1e-9df9-1bb2bcfa8c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:15.415566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15433c9-fef3-4f1e-9df9-1bb2bcfa8c39 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:15.433740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 723/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:17.722878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9c34-1445-46fa-8bea-e7454024a350 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:17.723501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9c34-1445-46fa-8bea-e7454024a350 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:17.723501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe9c34-1445-46fa-8bea-e7454024a350 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:17.724811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:17.731890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d9b31-629e-46e0-a9e2-f474169af114 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:17.732590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d9b31-629e-46e0-a9e2-f474169af114 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:17.732766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9d9b31-629e-46e0-a9e2-f474169af114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:17.808874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cc9d9b31-629e-46e0-a9e2-f474169af114 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:17.810051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cc9d9b31-629e-46e0-a9e2-f474169af114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:17.823541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:17.915145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:18.192967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe9c34-1445-46fa-8bea-e7454024a350 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:18.195581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5cbe9c34-1445-46fa-8bea-e7454024a350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:18.319227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe9c34-1445-46fa-8bea-e7454024a350 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:18.332111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:18.403826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80e9f48-0adf-41dc-9ead-50afea9a1733 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:18.404432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e9f48-0adf-41dc-9ead-50afea9a1733 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:18.404706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e9f48-0adf-41dc-9ead-50afea9a1733 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:18.404976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80e9f48-0adf-41dc-9ead-50afea9a1733 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:18.406639 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a80e9f48-0adf-41dc-9ead-50afea9a1733 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:18.537827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e9f48-0adf-41dc-9ead-50afea9a1733 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:18.539299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80e9f48-0adf-41dc-9ead-50afea9a1733 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:18.558719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 722/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:21.088973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66f38-1a14-44e0-9f4b-f284f9585096 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:21.089531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66f38-1a14-44e0-9f4b-f284f9585096 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:21.089531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba66f38-1a14-44e0-9f4b-f284f9585096 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:21.090944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:21.096681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7ccce-c0c6-4051-8114-de016f9554a5 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:21.097094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7ccce-c0c6-4051-8114-de016f9554a5 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:21.097194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b7ccce-c0c6-4051-8114-de016f9554a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:21.145980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:21.166055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a4b7ccce-c0c6-4051-8114-de016f9554a5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:21.167271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a4b7ccce-c0c6-4051-8114-de016f9554a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:21.266013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:21.546194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66f38-1a14-44e0-9f4b-f284f9585096 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:21.547801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4ba66f38-1a14-44e0-9f4b-f284f9585096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:21.657443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba66f38-1a14-44e0-9f4b-f284f9585096 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:21.697954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:21.734481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fab3ab-818d-4dca-9377-0682bd983edd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:21.735095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab3ab-818d-4dca-9377-0682bd983edd service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:21.735368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab3ab-818d-4dca-9377-0682bd983edd service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:21.735636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fab3ab-818d-4dca-9377-0682bd983edd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:21.737236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-31fab3ab-818d-4dca-9377-0682bd983edd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:21.980439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fab3ab-818d-4dca-9377-0682bd983edd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:21.981900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fab3ab-818d-4dca-9377-0682bd983edd service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:22.000080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 726/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:24.256326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba373c-82de-4f9c-ba63-186fffcdea70 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:24.257224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba373c-82de-4f9c-ba63-186fffcdea70 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:24.257224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba373c-82de-4f9c-ba63-186fffcdea70 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:24.259538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:24.268334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bff4a-97fc-4c85-a6d9-33fb38466d09 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:24.269077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bff4a-97fc-4c85-a6d9-33fb38466d09 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:24.269266 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bff4a-97fc-4c85-a6d9-33fb38466d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:24.342131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4b5bff4a-97fc-4c85-a6d9-33fb38466d09 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:24.343506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4b5bff4a-97fc-4c85-a6d9-33fb38466d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:24.367410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:24.456094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:24.737628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba373c-82de-4f9c-ba63-186fffcdea70 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:24.739243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-23ba373c-82de-4f9c-ba63-186fffcdea70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:24.850452 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba373c-82de-4f9c-ba63-186fffcdea70 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:24.872333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:24.931787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaf3e7-02e3-421e-a61f-eb0b872bf99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:24.932417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaf3e7-02e3-421e-a61f-eb0b872bf99e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:24.932706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaf3e7-02e3-421e-a61f-eb0b872bf99e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:24.932965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaf3e7-02e3-421e-a61f-eb0b872bf99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:24.934586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-17eaf3e7-02e3-421e-a61f-eb0b872bf99e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:25.051372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaf3e7-02e3-421e-a61f-eb0b872bf99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:25.052809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaf3e7-02e3-421e-a61f-eb0b872bf99e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:25.071043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 725/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:05:27.404335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55d287-b8de-4817-a276-fcb0f13d5ce2 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:27.405330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55d287-b8de-4817-a276-fcb0f13d5ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:27.405330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce55d287-b8de-4817-a276-fcb0f13d5ce2 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:27.406825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:27.415123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0f805-903d-4dea-b87d-5e4e6635f5f2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:27.415551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0f805-903d-4dea-b87d-5e4e6635f5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:27.415653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b0f805-903d-4dea-b87d-5e4e6635f5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:27.482141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:27.486059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c8b0f805-903d-4dea-b87d-5e4e6635f5f2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:27.488771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c8b0f805-903d-4dea-b87d-5e4e6635f5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:27.593487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:27.884711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55d287-b8de-4817-a276-fcb0f13d5ce2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:27.886292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ce55d287-b8de-4817-a276-fcb0f13d5ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:27.993801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce55d287-b8de-4817-a276-fcb0f13d5ce2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:28.046681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:28.071727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3bea12-ab0a-468c-9949-42975e950c36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:28.072358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bea12-ab0a-468c-9949-42975e950c36 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:28.072636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bea12-ab0a-468c-9949-42975e950c36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:28.072895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3bea12-ab0a-468c-9949-42975e950c36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:28.074538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0a3bea12-ab0a-468c-9949-42975e950c36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:28.203980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bea12-ab0a-468c-9949-42975e950c36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:28.205425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3bea12-ab0a-468c-9949-42975e950c36 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:28.228769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 729/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:30.442523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b569e5c-36c7-4d42-af8b-19f0ff8513ab None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:30.443102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b569e5c-36c7-4d42-af8b-19f0ff8513ab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:30.443102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b569e5c-36c7-4d42-af8b-19f0ff8513ab None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:30.444388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:30.450967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349b68-98e1-4cf6-9f9a-25bff32a3863 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:30.451626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349b68-98e1-4cf6-9f9a-25bff32a3863 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:30.451862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349b68-98e1-4cf6-9f9a-25bff32a3863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:30.525198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e2349b68-98e1-4cf6-9f9a-25bff32a3863 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:30.526449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e2349b68-98e1-4cf6-9f9a-25bff32a3863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:30.555901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:30.625479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:30.916562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b569e5c-36c7-4d42-af8b-19f0ff8513ab None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:30.918950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8b569e5c-36c7-4d42-af8b-19f0ff8513ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:31.041455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b569e5c-36c7-4d42-af8b-19f0ff8513ab None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:31.076995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:31.125730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23f93e-e6fe-4a7f-9cb6-aec2b7eabe9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:31.126229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23f93e-e6fe-4a7f-9cb6-aec2b7eabe9c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:31.126503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23f93e-e6fe-4a7f-9cb6-aec2b7eabe9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:31.126764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23f93e-e6fe-4a7f-9cb6-aec2b7eabe9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:31.128355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3d23f93e-e6fe-4a7f-9cb6-aec2b7eabe9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:31.246267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23f93e-e6fe-4a7f-9cb6-aec2b7eabe9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:31.247723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23f93e-e6fe-4a7f-9cb6-aec2b7eabe9c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:31.266562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 728/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:33.456906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611856e-fb35-4e97-90a7-05dad2983009 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:33.457508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611856e-fb35-4e97-90a7-05dad2983009 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:33.457508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611856e-fb35-4e97-90a7-05dad2983009 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:33.458811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:33.465144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c8a80-47d1-4fc3-9fbe-b322cc8957e2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:33.465946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c8a80-47d1-4fc3-9fbe-b322cc8957e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:33.466096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c8a80-47d1-4fc3-9fbe-b322cc8957e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:33.538133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-126c8a80-47d1-4fc3-9fbe-b322cc8957e2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:33.539343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-126c8a80-47d1-4fc3-9fbe-b322cc8957e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:33.547896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:33.643876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:33.934597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611856e-fb35-4e97-90a7-05dad2983009 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:33.936920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4611856e-fb35-4e97-90a7-05dad2983009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:34.045340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611856e-fb35-4e97-90a7-05dad2983009 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:34.105196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:34.129303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9adda-8e2b-4873-97de-ef3b4cb6bd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:34.129883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9adda-8e2b-4873-97de-ef3b4cb6bd25 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:34.130164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9adda-8e2b-4873-97de-ef3b4cb6bd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:34.130425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9adda-8e2b-4873-97de-ef3b4cb6bd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:34.132033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8db9adda-8e2b-4873-97de-ef3b4cb6bd25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:34.244175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9adda-8e2b-4873-97de-ef3b4cb6bd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:34.245670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db9adda-8e2b-4873-97de-ef3b4cb6bd25 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:34.264986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 732/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:36.519555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e086-29ae-4967-8d53-accc645ddfdb None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:36.520042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e086-29ae-4967-8d53-accc645ddfdb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:36.520153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836e086-29ae-4967-8d53-accc645ddfdb None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:36.521501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:36.527159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8742f-da91-41e5-b51f-2e92574c2c2f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:36.527622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8742f-da91-41e5-b51f-2e92574c2c2f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:36.527733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8742f-da91-41e5-b51f-2e92574c2c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:36.601171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3df8742f-da91-41e5-b51f-2e92574c2c2f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:36.602426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3df8742f-da91-41e5-b51f-2e92574c2c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:36.643292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:36.706810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:36.995291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836e086-29ae-4967-8d53-accc645ddfdb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:36.997560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e836e086-29ae-4967-8d53-accc645ddfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:37.116599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836e086-29ae-4967-8d53-accc645ddfdb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:37.158362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:37.195885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c4f87-7d4c-49e7-863d-5c037d689754 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:37.196520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c4f87-7d4c-49e7-863d-5c037d689754 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:37.196797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c4f87-7d4c-49e7-863d-5c037d689754 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:37.197066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c4f87-7d4c-49e7-863d-5c037d689754 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:37.198706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d86c4f87-7d4c-49e7-863d-5c037d689754 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:37.314951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c4f87-7d4c-49e7-863d-5c037d689754 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:37.316363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c4f87-7d4c-49e7-863d-5c037d689754 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:37.335483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 731/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:39.486933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32183cde-b765-4a73-b79b-66fd5a0b04f4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:39.487850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32183cde-b765-4a73-b79b-66fd5a0b04f4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:39.487850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32183cde-b765-4a73-b79b-66fd5a0b04f4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:39.490131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:39.495897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810a9bd-05c8-4b71-972c-1c152c8a9639 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:39.496341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810a9bd-05c8-4b71-972c-1c152c8a9639 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:39.496441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5810a9bd-05c8-4b71-972c-1c152c8a9639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:39.566213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5810a9bd-05c8-4b71-972c-1c152c8a9639 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:39.567432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5810a9bd-05c8-4b71-972c-1c152c8a9639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:39.586684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:39.683269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:39.968700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32183cde-b765-4a73-b79b-66fd5a0b04f4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:39.970360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-32183cde-b765-4a73-b79b-66fd5a0b04f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:40.077786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32183cde-b765-4a73-b79b-66fd5a0b04f4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:40.146651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:40.153606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9697b32-eb47-49be-8a37-739f52727c14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:40.154252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697b32-eb47-49be-8a37-739f52727c14 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:40.154525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697b32-eb47-49be-8a37-739f52727c14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:40.154791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9697b32-eb47-49be-8a37-739f52727c14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:40.156426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b9697b32-eb47-49be-8a37-739f52727c14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:40.271481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9697b32-eb47-49be-8a37-739f52727c14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:40.272828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9697b32-eb47-49be-8a37-739f52727c14 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:40.291554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 735/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:42.484315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6162f5-cfe5-4f3b-8e44-b46d409e4669 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:42.484956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6162f5-cfe5-4f3b-8e44-b46d409e4669 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:42.484956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6162f5-cfe5-4f3b-8e44-b46d409e4669 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:42.486261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:05:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:05:42.492942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892ff9a-22df-47e2-8364-27fafdee44c8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:42.493417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892ff9a-22df-47e2-8364-27fafdee44c8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:42.493526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892ff9a-22df-47e2-8364-27fafdee44c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:42.562883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8892ff9a-22df-47e2-8364-27fafdee44c8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:05:42.564072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8892ff9a-22df-47e2-8364-27fafdee44c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:42.612998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:42.663124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:05:42.939802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6162f5-cfe5-4f3b-8e44-b46d409e4669 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:05:42.941354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ce6162f5-cfe5-4f3b-8e44-b46d409e4669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:43.052276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6162f5-cfe5-4f3b-8e44-b46d409e4669 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:05:43.103953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:43.130955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be8efce-911b-42ed-ab82-911fb6a12bf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:05:43.131563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8efce-911b-42ed-ab82-911fb6a12bf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:05:43.131832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8efce-911b-42ed-ab82-911fb6a12bf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:05:43.132093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be8efce-911b-42ed-ab82-911fb6a12bf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:05:43.133668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6be8efce-911b-42ed-ab82-911fb6a12bf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:05:43.254554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8efce-911b-42ed-ab82-911fb6a12bf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:05:43.256748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be8efce-911b-42ed-ab82-911fb6a12bf1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:05:43.282127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 734/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:05:52.661621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=22 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:05:53.279469 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=66 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:03.054375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b675-327d-4492-8f37-a73ab084f5fd None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:03.055008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b675-327d-4492-8f37-a73ab084f5fd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:03.055008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626b675-327d-4492-8f37-a73ab084f5fd None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:03.057026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 737/1471] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:06:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:06:03.063909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99961c-4cd2-44fd-842b-2f7a63deb505 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:03.064398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99961c-4cd2-44fd-842b-2f7a63deb505 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:03.064517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99961c-4cd2-44fd-842b-2f7a63deb505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:03.133310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-be99961c-4cd2-44fd-842b-2f7a63deb505 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:06:03.134569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-be99961c-4cd2-44fd-842b-2f7a63deb505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:03.250861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 735/1472] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:06:03.545373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1cec8-dbb6-4b76-8ca6-1b6a380d3f1c None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:03.545842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1cec8-dbb6-4b76-8ca6-1b6a380d3f1c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:03.546119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e1cec8-dbb6-4b76-8ca6-1b6a380d3f1c None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:03.547488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 738/1473] 10.222.0.23 () {54 vars in 962 bytes} [Wed Oct 5 14:06:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:06:03.552594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6d007-3567-4a46-b2d7-df9badd07723 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:03.552881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6d007-3567-4a46-b2d7-df9badd07723 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:03.553122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6d007-3567-4a46-b2d7-df9badd07723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:03.632574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d4c6d007-3567-4a46-b2d7-df9badd07723 None None] MFA Rules not processed for user `417b661f770846f189c36d0804b6dce5`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:06:03.633837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d4c6d007-3567-4a46-b2d7-df9badd07723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:03.656326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:03.716204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 736/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Wed Oct 5 14:06:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:06:03.724821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1cec8-dbb6-4b76-8ca6-1b6a380d3f1c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:06:03.726439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b3e1cec8-dbb6-4b76-8ca6-1b6a380d3f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:03.850981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e1cec8-dbb6-4b76-8ca6-1b6a380d3f1c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:06:03.934341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b1d30-0533-4ed3-82ed-4a59275f706d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417b661f770846f189c36d0804b6dce5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:06:03.934937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b1d30-0533-4ed3-82ed-4a59275f706d service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:03.935206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b1d30-0533-4ed3-82ed-4a59275f706d service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:03.935465 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b1d30-0533-4ed3-82ed-4a59275f706d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:03.937152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fa7b1d30-0533-4ed3-82ed-4a59275f706d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:03.995794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:04.054749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b1d30-0533-4ed3-82ed-4a59275f706d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:06:04.056185 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b1d30-0533-4ed3-82ed-4a59275f706d service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:06:04.074792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 739/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:06:06.300356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785b776-6d69-45c6-bbc7-0e24aac42138 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:06.301229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785b776-6d69-45c6-bbc7-0e24aac42138 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:06.301229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5785b776-6d69-45c6-bbc7-0e24aac42138 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:06.303448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 737/1476] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:06:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:06:06.308929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42e1d5-0c09-436c-be07-8db0804be363 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:06.309377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42e1d5-0c09-436c-be07-8db0804be363 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:06.309488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a42e1d5-0c09-436c-be07-8db0804be363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:06.377179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4a42e1d5-0c09-436c-be07-8db0804be363 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:06:06.378409 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4a42e1d5-0c09-436c-be07-8db0804be363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:06.494762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 740/1477] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:06:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:06:06.767870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785b776-6d69-45c6-bbc7-0e24aac42138 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:06:06.770508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5785b776-6d69-45c6-bbc7-0e24aac42138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:06.910162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5785b776-6d69-45c6-bbc7-0e24aac42138 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:06:06.961776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:07.008205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd787d2-a1fd-419c-a1ef-15d398856056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:06:07.008838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd787d2-a1fd-419c-a1ef-15d398856056 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:07.009114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd787d2-a1fd-419c-a1ef-15d398856056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:07.009383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd787d2-a1fd-419c-a1ef-15d398856056 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:07.010055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd787d2-a1fd-419c-a1ef-15d398856056 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:06:07.011793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-dcd787d2-a1fd-419c-a1ef-15d398856056 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:06:07.011793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Oct 05 14:06:07.011793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: Oct 05 14:06:07.012927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd787d2-a1fd-419c-a1ef-15d398856056 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:06:07.032145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 738/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Wed Oct 5 14:06:06 2022] GET /identity/v3/projects => generated 2553 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:06:09.223412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744df3be-777d-43d5-afc2-7b79e868a3dc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:09.224336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744df3be-777d-43d5-afc2-7b79e868a3dc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:09.224336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744df3be-777d-43d5-afc2-7b79e868a3dc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:09.226542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 741/1479] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:06:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:06:09.236171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b153f-881c-40e2-8298-b7c3648a20f8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:09.236647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b153f-881c-40e2-8298-b7c3648a20f8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:09.236788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b153f-881c-40e2-8298-b7c3648a20f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:09.316084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5b3b153f-881c-40e2-8298-b7c3648a20f8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:06:09.317274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5b3b153f-881c-40e2-8298-b7c3648a20f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:09.421294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 739/1480] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:06:09.718639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744df3be-777d-43d5-afc2-7b79e868a3dc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:06:09.720233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-744df3be-777d-43d5-afc2-7b79e868a3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:09.730649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:09.882928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744df3be-777d-43d5-afc2-7b79e868a3dc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:06:09.976044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:10.014094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627b819-99e9-4307-8198-c738c47d1be9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:06:10.014740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627b819-99e9-4307-8198-c738c47d1be9 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:10.015004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627b819-99e9-4307-8198-c738c47d1be9 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:10.015248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627b819-99e9-4307-8198-c738c47d1be9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:10.016860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7627b819-99e9-4307-8198-c738c47d1be9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:10.110251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627b819-99e9-4307-8198-c738c47d1be9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:06:10.111830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627b819-99e9-4307-8198-c738c47d1be9 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:06:10.129683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 742/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:06:12.479194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6b6c-a595-453d-88d0-21a510c983f6 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:12.480129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6b6c-a595-453d-88d0-21a510c983f6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:12.480129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380d6b6c-a595-453d-88d0-21a510c983f6 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:12.482662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 740/1482] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:06:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:06:12.489006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f1c2-74fb-486e-820a-9a27a2e760af None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:12.489425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f1c2-74fb-486e-820a-9a27a2e760af None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:12.489516 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f4f1c2-74fb-486e-820a-9a27a2e760af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:12.565368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-60f4f1c2-74fb-486e-820a-9a27a2e760af None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:06:12.566647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-60f4f1c2-74fb-486e-820a-9a27a2e760af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:12.616486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:12.672232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 743/1483] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:06:12.970166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d6b6c-a595-453d-88d0-21a510c983f6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:06:12.971724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-380d6b6c-a595-453d-88d0-21a510c983f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:12.996815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:13.081578 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380d6b6c-a595-453d-88d0-21a510c983f6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:06:13.161954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d636ff-07bd-49d4-ae05-88d3d35b7446 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:06:13.162490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d636ff-07bd-49d4-ae05-88d3d35b7446 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:13.162763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d636ff-07bd-49d4-ae05-88d3d35b7446 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:13.163029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d636ff-07bd-49d4-ae05-88d3d35b7446 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:13.164622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-13d636ff-07bd-49d4-ae05-88d3d35b7446 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:13.246365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:13.292065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d636ff-07bd-49d4-ae05-88d3d35b7446 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:06:13.293510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d636ff-07bd-49d4-ae05-88d3d35b7446 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:06:13.314214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 741/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:06:15.499927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036285-d094-496d-af80-55e45250b98d None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:15.500799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036285-d094-496d-af80-55e45250b98d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:15.500799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6036285-d094-496d-af80-55e45250b98d None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:15.503943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 744/1485] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:06:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:06:15.511055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7843d6f-77f4-47a5-ba0d-17255d253b32 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:15.511511 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7843d6f-77f4-47a5-ba0d-17255d253b32 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:15.511733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7843d6f-77f4-47a5-ba0d-17255d253b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:15.581686 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f7843d6f-77f4-47a5-ba0d-17255d253b32 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:06:15.582923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f7843d6f-77f4-47a5-ba0d-17255d253b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:15.684039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 742/1486] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:06:15.978975 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6036285-d094-496d-af80-55e45250b98d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:06:15.981835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c6036285-d094-496d-af80-55e45250b98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:16.111042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6036285-d094-496d-af80-55e45250b98d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:06:16.191361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:16.193119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2a8f3e-1ea7-4708-86fb-024b295cb666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:06:16.193912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a8f3e-1ea7-4708-86fb-024b295cb666 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:16.194168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a8f3e-1ea7-4708-86fb-024b295cb666 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:16.194453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2a8f3e-1ea7-4708-86fb-024b295cb666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:16.198065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4f2a8f3e-1ea7-4708-86fb-024b295cb666 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:16.315746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a8f3e-1ea7-4708-86fb-024b295cb666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:06:16.317101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2a8f3e-1ea7-4708-86fb-024b295cb666 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:06:16.335531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 745/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:06:18.598615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db037f-295d-4ca0-ba5d-6f3e3504a4b8 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:18.599202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db037f-295d-4ca0-ba5d-6f3e3504a4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:18.599202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db037f-295d-4ca0-ba5d-6f3e3504a4b8 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:18.600529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 743/1488] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:06:18.607832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1c57-fd8a-4fe9-9ff0-adc02bfd98d6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:18.608315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1c57-fd8a-4fe9-9ff0-adc02bfd98d6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:18.608420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd1c57-fd8a-4fe9-9ff0-adc02bfd98d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:18.695332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-60cd1c57-fd8a-4fe9-9ff0-adc02bfd98d6 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:06:18.696580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-60cd1c57-fd8a-4fe9-9ff0-adc02bfd98d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:18.751897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:18.800037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 746/1489] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:06:19.091816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db037f-295d-4ca0-ba5d-6f3e3504a4b8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:06:19.093383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-97db037f-295d-4ca0-ba5d-6f3e3504a4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:19.133247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:19.209282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db037f-295d-4ca0-ba5d-6f3e3504a4b8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:06:19.284359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff33e14-4e81-4eb6-b7dd-47f3dbe7e118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:06:19.285043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff33e14-4e81-4eb6-b7dd-47f3dbe7e118 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:19.285322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff33e14-4e81-4eb6-b7dd-47f3dbe7e118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:19.285666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff33e14-4e81-4eb6-b7dd-47f3dbe7e118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:19.288533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dff33e14-4e81-4eb6-b7dd-47f3dbe7e118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:19.376292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:19.408651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff33e14-4e81-4eb6-b7dd-47f3dbe7e118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:06:19.410188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff33e14-4e81-4eb6-b7dd-47f3dbe7e118 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:06:19.428638 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 744/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:06:21.791771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40d4b-41fe-4c24-98e1-911608a341f8 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:21.792634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40d4b-41fe-4c24-98e1-911608a341f8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:21.792634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a40d4b-41fe-4c24-98e1-911608a341f8 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:21.794741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 747/1491] 10.222.0.23 () {54 vars in 927 bytes} [Wed Oct 5 14:06:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:06:21.803815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa655b5-7d5b-4a7c-8398-f3f1736a318e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:21.804251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa655b5-7d5b-4a7c-8398-f3f1736a318e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:21.804345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa655b5-7d5b-4a7c-8398-f3f1736a318e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:21.890611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9aa655b5-7d5b-4a7c-8398-f3f1736a318e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:06:21.892227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9aa655b5-7d5b-4a7c-8398-f3f1736a318e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:22.003441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 745/1492] 10.222.0.23 () {58 vars in 1024 bytes} [Wed Oct 5 14:06:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:06:22.298467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a40d4b-41fe-4c24-98e1-911608a341f8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:06:22.299998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39a40d4b-41fe-4c24-98e1-911608a341f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:22.406655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a40d4b-41fe-4c24-98e1-911608a341f8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:06:22.484964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3573f976-a750-49a1-b834-dee95b7e4828 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:06:22.485569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573f976-a750-49a1-b834-dee95b7e4828 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:06:22.485871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573f976-a750-49a1-b834-dee95b7e4828 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:06:22.486134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3573f976-a750-49a1-b834-dee95b7e4828 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:06:22.487752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3573f976-a750-49a1-b834-dee95b7e4828 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:06:22.497141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:22.603835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573f976-a750-49a1-b834-dee95b7e4828 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:06:22.605192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3573f976-a750-49a1-b834-dee95b7e4828 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:06:22.624328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 748/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:06:32.001949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=82 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:06:32.621900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=50 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:42.286954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79277ca-3994-441c-aa3d-331938aec2e8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:42.287618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79277ca-3994-441c-aa3d-331938aec2e8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:42.287618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79277ca-3994-441c-aa3d-331938aec2e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:42.368442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a79277ca-3994-441c-aa3d-331938aec2e8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:07:42.369665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a79277ca-3994-441c-aa3d-331938aec2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:42.478117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 746/1494] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:07:42.487612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11039bed-f7e1-416b-8a2c-36eda5061b87 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:42.488173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11039bed-f7e1-416b-8a2c-36eda5061b87 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:42.488173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11039bed-f7e1-416b-8a2c-36eda5061b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:42.572688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-11039bed-f7e1-416b-8a2c-36eda5061b87 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:07:42.573872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11039bed-f7e1-416b-8a2c-36eda5061b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:42.680837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 749/1495] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:07:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:07:42.685728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79277ca-3994-441c-aa3d-331938aec2e8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:42.686925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a79277ca-3994-441c-aa3d-331938aec2e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:42.699150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79277ca-3994-441c-aa3d-331938aec2e8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:42.771223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:42.807100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:42.807709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:42.807982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:42.808246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:42.808878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:07:42.809178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:42.810223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:07:42.810223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Oct 05 14:07:42.810223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: Oct 05 14:07:42.811105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:42.823402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 747/1496] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:07:42.831891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11039bed-f7e1-416b-8a2c-36eda5061b87 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:42.842468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11039bed-f7e1-416b-8a2c-36eda5061b87 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:42.915707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:42.954074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:42.954715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:42.954993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:42.955254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:42.955897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:42.957754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:07:42.957754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Oct 05 14:07:42.957754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: Oct 05 14:07:42.958956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:43.030448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 750/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:07:42 2022] POST /identity/v3/projects => generated 377 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:07:43.036502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:43.038127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:43.073203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27485c0-42a3-4a3a-86b2-aeb1c11b7523 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:43.181603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:43.182250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:43.182516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:43.182774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:43.183394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:43.184406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:07:43.184406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Oct 05 14:07:43.184406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: Oct 05 14:07:43.185598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:43.229403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 748/1498] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:07:43 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:07:43.234154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:43.235677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:43.300947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993953d-0ea1-40b5-a834-29f0d0eded6a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:43.391852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:43.404238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:43.404890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:43.405127 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:43.405392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:43.406181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:43.407217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:07:43.407217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Oct 05 14:07:43.407217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: Oct 05 14:07:43.408110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:43.420788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 751/1499] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:07:43.425140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:43.426783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:43.438423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94cce0f-74dc-4320-bb20-c28b69fe06ed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:43.463989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:43.535407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebd5643-2869-444b-9290-9c0870d646aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:43.536019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd5643-2869-444b-9290-9c0870d646aa admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:43.536314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd5643-2869-444b-9290-9c0870d646aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:43.536553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebd5643-2869-444b-9290-9c0870d646aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:43.537190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd5643-2869-444b-9290-9c0870d646aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:43.538270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-8ebd5643-2869-444b-9290-9c0870d646aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:07:43.538270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Oct 05 14:07:43.538270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: Oct 05 14:07:43.539181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebd5643-2869-444b-9290-9c0870d646aa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:43.551653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 749/1500] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:07:43.558113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:43.560296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:43.602728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06582d7e-1f76-42b2-96a0-b41a611d262c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:43.701683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c55fb-085c-4e27-9e90-b09069710431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:43.702288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c55fb-085c-4e27-9e90-b09069710431 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:43.702559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c55fb-085c-4e27-9e90-b09069710431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:43.702824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c55fb-085c-4e27-9e90-b09069710431 admin admin] PATH_INFO: `/v3/projects/48795f58c7d043c9b0a7ac99bbef5f70/users/61a2e4dc22424e5cb74d1c3048e20582/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:43.726365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c55fb-085c-4e27-9e90-b09069710431 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48795f58c7d043c9b0a7ac99bbef5f70, user_id=61a2e4dc22424e5cb74d1c3048e20582, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:43.727382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-174c55fb-085c-4e27-9e90-b09069710431 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:07:43.727382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Oct 05 14:07:43.727382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: Oct 05 14:07:43.729309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c55fb-085c-4e27-9e90-b09069710431 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:43.756680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 752/1501] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:07:43 2022] PUT /identity/v3/projects/48795f58c7d043c9b0a7ac99bbef5f70/users/61a2e4dc22424e5cb74d1c3048e20582/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:07:43.769420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819dc32-d844-4ca6-9b6a-4d69924ccf9e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:43.769756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819dc32-d844-4ca6-9b6a-4d69924ccf9e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:43.770025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e819dc32-d844-4ca6-9b6a-4d69924ccf9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:43.829262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e819dc32-d844-4ca6-9b6a-4d69924ccf9e None None] MFA Rules not processed for user `61a2e4dc22424e5cb74d1c3048e20582`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:07:43.830493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e819dc32-d844-4ca6-9b6a-4d69924ccf9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:43.908642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 750/1502] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:07:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:07:43.922598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e991-9324-4425-aa17-3698e9459fe4 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:43.922879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e991-9324-4425-aa17-3698e9459fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:43.923141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365e991-9324-4425-aa17-3698e9459fe4 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:43.924510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 753/1503] 10.222.0.23 () {54 vars in 963 bytes} [Wed Oct 5 14:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:07:43.929568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204e5af-4e2d-4ca5-9528-7a996d6a4767 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:43.929892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204e5af-4e2d-4ca5-9528-7a996d6a4767 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:43.930155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204e5af-4e2d-4ca5-9528-7a996d6a4767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:43.962195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:44.014450 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7204e5af-4e2d-4ca5-9528-7a996d6a4767 None None] MFA Rules not processed for user `44fba9b1749b4172bc9479ca980fcf48`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:07:44.015652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7204e5af-4e2d-4ca5-9528-7a996d6a4767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:44.092897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 751/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Wed Oct 5 14:07:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:07:44.102614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365e991-9324-4425-aa17-3698e9459fe4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:44.104221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8365e991-9324-4425-aa17-3698e9459fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:44.159694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:44.216373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8365e991-9324-4425-aa17-3698e9459fe4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:44.291189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:44.291789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:44.292060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:44.292316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:44.293925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:44.400552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:44.402024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:44.405430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:44.420947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 754/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:07:44 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:07:44.470112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e7c3a-aabd-418a-adc8-704b8fad92a0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:44.470558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e7c3a-aabd-418a-adc8-704b8fad92a0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:44.470558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8e7c3a-aabd-418a-adc8-704b8fad92a0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:44.471822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 752/1506] 10.222.0.23 () {52 vars in 830 bytes} [Wed Oct 5 14:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:07:44.475867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:44.477092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:44.594871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291196ee-13be-4f2e-9c5c-6e2bb355405d service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:44.703127 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:44.703798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:44.704047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:44.704315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] PATH_INFO: `/v3/users/61a2e4dc22424e5cb74d1c3048e20582` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:44.717673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61a2e4dc22424e5cb74d1c3048e20582)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:44.718746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:07:44.718746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Oct 05 14:07:44.718746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: Oct 05 14:07:44.719965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:44.729903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:44.792468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61a2e4dc22424e5cb74d1c3048e20582'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:44.806315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61a2e4dc22424e5cb74d1c3048e20582'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:44.806758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61a2e4dc22424e5cb74d1c3048e20582'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:44.818341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61a2e4dc22424e5cb74d1c3048e20582'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:44.843705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61a2e4dc22424e5cb74d1c3048e20582'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:44.850120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 755/1507] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:07:44 2022] DELETE /identity/v3/users/61a2e4dc22424e5cb74d1c3048e20582 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:07:44.854887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132eb19-e51c-4bc3-95c9-4ee1cc17ce79 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:44.855165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132eb19-e51c-4bc3-95c9-4ee1cc17ce79 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:44.855429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132eb19-e51c-4bc3-95c9-4ee1cc17ce79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:44.913766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4132eb19-e51c-4bc3-95c9-4ee1cc17ce79 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:07:44.914976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4132eb19-e51c-4bc3-95c9-4ee1cc17ce79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:44.991003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:45.009069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 753/1508] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:07:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:07:45.019549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:45.020790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:45.124675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a78ad1f-8131-442e-96b0-585db0eddc58 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:45.183279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:45.204453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a46bc-9412-40f0-918a-295dd0af5e94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:45.205083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a46bc-9412-40f0-918a-295dd0af5e94 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:45.205356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a46bc-9412-40f0-918a-295dd0af5e94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:45.205623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a46bc-9412-40f0-918a-295dd0af5e94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:45.207380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-211a46bc-9412-40f0-918a-295dd0af5e94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:45.319396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a46bc-9412-40f0-918a-295dd0af5e94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:45.320749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a46bc-9412-40f0-918a-295dd0af5e94 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:45.339283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 756/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:07:45.700595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132eb19-e51c-4bc3-95c9-4ee1cc17ce79 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:07:45.701942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4132eb19-e51c-4bc3-95c9-4ee1cc17ce79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:07:45.713695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132eb19-e51c-4bc3-95c9-4ee1cc17ce79 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:07:45.809993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:07:45.810641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:07:45.810913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:07:45.811179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] PATH_INFO: `/v3/projects/48795f58c7d043c9b0a7ac99bbef5f70` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:07:45.820931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48795f58c7d043c9b0a7ac99bbef5f70)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:07:45.822017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:07:45.822017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Oct 05 14:07:45.822017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: Oct 05 14:07:45.823230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:07:45.886219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48795f58c7d043c9b0a7ac99bbef5f70', 'request_id': 'req-47c7b437-197a-4d4f-a649-c847ab87d1a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:45.893913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48795f58c7d043c9b0a7ac99bbef5f70', 'request_id': 'req-47c7b437-197a-4d4f-a649-c847ab87d1a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:45.906925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48795f58c7d043c9b0a7ac99bbef5f70', 'request_id': 'req-47c7b437-197a-4d4f-a649-c847ab87d1a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:45.933118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] The token cache is being invalidate because project 48795f58c7d043c9b0a7ac99bbef5f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:07:45.933404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:45.933993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-47c7b437-197a-4d4f-a649-c847ab87d1a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:07:45.935392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 754/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:07:45 2022] DELETE /identity/v3/projects/48795f58c7d043c9b0a7ac99bbef5f70 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:07:55.336904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=62 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:07:55.931435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:09:14.595590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f802-1672-4549-bbb5-7f77e44f90bc None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:14.596395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f802-1672-4549-bbb5-7f77e44f90bc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:14.596395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f47f802-1672-4549-bbb5-7f77e44f90bc None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:14.598627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:09:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:14.603847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4318a-b16d-45a7-b0e9-9164da8fac93 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:14.604343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4318a-b16d-45a7-b0e9-9164da8fac93 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:14.604343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4318a-b16d-45a7-b0e9-9164da8fac93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:14.693829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-10c4318a-b16d-45a7-b0e9-9164da8fac93 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:14.695030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10c4318a-b16d-45a7-b0e9-9164da8fac93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:14.822461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:14.977107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f802-1672-4549-bbb5-7f77e44f90bc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:14.979791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3f47f802-1672-4549-bbb5-7f77e44f90bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:15.104844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f47f802-1672-4549-bbb5-7f77e44f90bc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:15.111110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229cf2ad-b9f7-4382-918f-f519bf6d898b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:15.111724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf2ad-b9f7-4382-918f-f519bf6d898b service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:15.112002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf2ad-b9f7-4382-918f-f519bf6d898b service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:15.112274 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229cf2ad-b9f7-4382-918f-f519bf6d898b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:15.121020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229cf2ad-b9f7-4382-918f-f519bf6d898b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:15.122685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229cf2ad-b9f7-4382-918f-f519bf6d898b service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:15.132152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 758/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:09:16.151378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92269def-5d14-4d24-822f-13ade63c1aac None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:16.152005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92269def-5d14-4d24-822f-13ade63c1aac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:16.152005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92269def-5d14-4d24-822f-13ade63c1aac None None] PATH_INFO: `/v3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:16.153377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 756/1514] 10.222.0.44 () {54 vars in 925 bytes} [Wed Oct 5 14:09:16 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:09:16.162592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a653-74e0-46f6-bd3d-cc5a3b43bc84 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:16.163082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a653-74e0-46f6-bd3d-cc5a3b43bc84 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:16.163082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31a653-74e0-46f6-bd3d-cc5a3b43bc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:16.205966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe0ad4-ddeb-41e2-bd8e-873ce25b7205 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:16.206333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe0ad4-ddeb-41e2-bd8e-873ce25b7205 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:16.206666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe0ad4-ddeb-41e2-bd8e-873ce25b7205 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:16.208089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 757/1515] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:16.214406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7e04f-3e35-4d3c-ac6b-4d03a2a5c1ef None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:16.214764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7e04f-3e35-4d3c-ac6b-4d03a2a5c1ef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:16.215037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab7e04f-3e35-4d3c-ac6b-4d03a2a5c1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:16.229224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6d31a653-74e0-46f6-bd3d-cc5a3b43bc84 None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:16.230644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6d31a653-74e0-46f6-bd3d-cc5a3b43bc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:16.236333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eab7e04f-3e35-4d3c-ac6b-4d03a2a5c1ef None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:16.237489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eab7e04f-3e35-4d3c-ac6b-4d03a2a5c1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:16.245298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 758/1516] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:16.299173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:09:16.304822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 759/1517] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Oct 5 14:09:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:16.395925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7e04f-3e35-4d3c-ac6b-4d03a2a5c1ef None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:16.418137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab7e04f-3e35-4d3c-ac6b-4d03a2a5c1ef None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:16.425623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffbea26-b38a-4eae-b392-34feab8fc859 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:16.426260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbea26-b38a-4eae-b392-34feab8fc859 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:16.426531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbea26-b38a-4eae-b392-34feab8fc859 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:16.426815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffbea26-b38a-4eae-b392-34feab8fc859 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:16.435438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbea26-b38a-4eae-b392-34feab8fc859 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:16.437060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffbea26-b38a-4eae-b392-34feab8fc859 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:16.448613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 759/1518] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:09:25.000993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8df98-8715-4735-a9dc-c1ca3101a3e6 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:25.001485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8df98-8715-4735-a9dc-c1ca3101a3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:25.001485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e8df98-8715-4735-a9dc-c1ca3101a3e6 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:25.002925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 760/1519] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 14:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:25.008030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cc8db-1a67-4999-8019-e59364f287c1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:25.008454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cc8db-1a67-4999-8019-e59364f287c1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:25.008574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cc8db-1a67-4999-8019-e59364f287c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:25.028957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2d9cc8db-1a67-4999-8019-e59364f287c1 None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:25.030208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2d9cc8db-1a67-4999-8019-e59364f287c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:25.040172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 760/1520] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 14:09:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:25.048075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8df98-8715-4735-a9dc-c1ca3101a3e6 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:25.063151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e8df98-8715-4735-a9dc-c1ca3101a3e6 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:25.068565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983af0b-7f9d-4de1-b347-2b165dc6b773 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:25.069436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983af0b-7f9d-4de1-b347-2b165dc6b773 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:25.069742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983af0b-7f9d-4de1-b347-2b165dc6b773 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:25.070008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983af0b-7f9d-4de1-b347-2b165dc6b773 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:25.078338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983af0b-7f9d-4de1-b347-2b165dc6b773 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:25.079968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983af0b-7f9d-4de1-b347-2b165dc6b773 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:25.089408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 761/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:09:27.947229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eec0b-8879-4ac1-8654-09f0fd98bf4b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:27.947792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eec0b-8879-4ac1-8654-09f0fd98bf4b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:27.947792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36eec0b-8879-4ac1-8654-09f0fd98bf4b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:27.949126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 761/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:27.956653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474d59d-c1fb-4d5c-907c-5f25b930cec7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:27.957142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474d59d-c1fb-4d5c-907c-5f25b930cec7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:27.957237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474d59d-c1fb-4d5c-907c-5f25b930cec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:27.983173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e474d59d-c1fb-4d5c-907c-5f25b930cec7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:27.984371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e474d59d-c1fb-4d5c-907c-5f25b930cec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:27.993410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 762/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:28.148973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eec0b-8879-4ac1-8654-09f0fd98bf4b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:28.168081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36eec0b-8879-4ac1-8654-09f0fd98bf4b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:28.173996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c96d98-92ea-48c5-afa4-46ebd5cde17f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:28.174605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c96d98-92ea-48c5-afa4-46ebd5cde17f service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:28.174859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c96d98-92ea-48c5-afa4-46ebd5cde17f service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:28.175148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c96d98-92ea-48c5-afa4-46ebd5cde17f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:28.184884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c96d98-92ea-48c5-afa4-46ebd5cde17f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:28.186531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c96d98-92ea-48c5-afa4-46ebd5cde17f service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:28.197796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 762/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:09:29.257358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb272c-5859-4bb5-b060-44810ff9e286 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:29.258350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb272c-5859-4bb5-b060-44810ff9e286 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:29.258350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb272c-5859-4bb5-b060-44810ff9e286 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:29.260407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 763/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Oct 5 14:09:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:29.267825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54074a79-6550-4b7b-9644-03a56f9aea3b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:29.268282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54074a79-6550-4b7b-9644-03a56f9aea3b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:29.268393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54074a79-6550-4b7b-9644-03a56f9aea3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:29.288360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-54074a79-6550-4b7b-9644-03a56f9aea3b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:29.289571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-54074a79-6550-4b7b-9644-03a56f9aea3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:29.298723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 763/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Oct 5 14:09:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:29.451272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb272c-5859-4bb5-b060-44810ff9e286 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:29.476756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bb272c-5859-4bb5-b060-44810ff9e286 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:29.483702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2522088-d682-4675-9138-2b6cc44647e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:29.484308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522088-d682-4675-9138-2b6cc44647e7 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:29.484581 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522088-d682-4675-9138-2b6cc44647e7 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:29.484843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2522088-d682-4675-9138-2b6cc44647e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:29.493916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2522088-d682-4675-9138-2b6cc44647e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:29.495528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2522088-d682-4675-9138-2b6cc44647e7 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:29.505436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 764/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:09:33.565084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf880b-5dd5-4529-9740-b7fa347d2205 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:33.565678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf880b-5dd5-4529-9740-b7fa347d2205 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:33.565678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cf880b-5dd5-4529-9740-b7fa347d2205 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:33.566992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 764/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:33.573601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709ed71-6f44-4e2f-882b-58443b1e3f26 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:33.574356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709ed71-6f44-4e2f-882b-58443b1e3f26 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:33.574476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7709ed71-6f44-4e2f-882b-58443b1e3f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:33.599307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7709ed71-6f44-4e2f-882b-58443b1e3f26 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:33.600540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7709ed71-6f44-4e2f-882b-58443b1e3f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:33.610207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 765/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:33.891922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf880b-5dd5-4529-9740-b7fa347d2205 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:33.893959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-28cf880b-5dd5-4529-9740-b7fa347d2205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:33.920871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:09:33.974326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cf880b-5dd5-4529-9740-b7fa347d2205 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:33.978160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a7d2ce-b6e0-4605-acc1-14a21def7a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:33.978781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7d2ce-b6e0-4605-acc1-14a21def7a8f service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:33.979053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7d2ce-b6e0-4605-acc1-14a21def7a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:33.979676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a7d2ce-b6e0-4605-acc1-14a21def7a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:33.989971 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7d2ce-b6e0-4605-acc1-14a21def7a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:33.991467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a7d2ce-b6e0-4605-acc1-14a21def7a8f service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:34.001035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 765/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:09:36.133453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed5eb0-3508-4329-b7f5-2a4743a4bc29 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:36.134063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed5eb0-3508-4329-b7f5-2a4743a4bc29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:36.134063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed5eb0-3508-4329-b7f5-2a4743a4bc29 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:36.135321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 766/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:36.141963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5640ba-cd4a-4da4-904d-3307d234f5ce None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:36.142444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5640ba-cd4a-4da4-904d-3307d234f5ce None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:36.142539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5640ba-cd4a-4da4-904d-3307d234f5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:36.170556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ef5640ba-cd4a-4da4-904d-3307d234f5ce None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:36.171774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ef5640ba-cd4a-4da4-904d-3307d234f5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:36.180901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 766/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:36.474368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed5eb0-3508-4329-b7f5-2a4743a4bc29 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:36.492435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed5eb0-3508-4329-b7f5-2a4743a4bc29 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:36.497050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391321d9-ec88-403f-a905-577d4cb7a95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:36.497661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391321d9-ec88-403f-a905-577d4cb7a95c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:36.497972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391321d9-ec88-403f-a905-577d4cb7a95c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:36.498240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391321d9-ec88-403f-a905-577d4cb7a95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:36.508056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391321d9-ec88-403f-a905-577d4cb7a95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:36.509509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391321d9-ec88-403f-a905-577d4cb7a95c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:36.519399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 767/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:09:40.106707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e4437-2902-4577-a24b-c42e7e1273ae None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:40.107621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e4437-2902-4577-a24b-c42e7e1273ae None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:40.107621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e4437-2902-4577-a24b-c42e7e1273ae None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:40.109790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 767/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:09:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:40.115769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332c2a-4aa5-4899-9bc5-7ee686b8cbde None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:40.116232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332c2a-4aa5-4899-9bc5-7ee686b8cbde None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:40.116339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91332c2a-4aa5-4899-9bc5-7ee686b8cbde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:40.138096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-91332c2a-4aa5-4899-9bc5-7ee686b8cbde None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:40.139356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-91332c2a-4aa5-4899-9bc5-7ee686b8cbde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:40.148980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 768/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:40.461929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e4437-2902-4577-a24b-c42e7e1273ae None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:40.464239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-947e4437-2902-4577-a24b-c42e7e1273ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:40.550249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e4437-2902-4577-a24b-c42e7e1273ae None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:40.554276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc1e4b5-c4ab-48fc-a186-907ed9187baa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:40.554909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e4b5-c4ab-48fc-a186-907ed9187baa service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:40.555189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e4b5-c4ab-48fc-a186-907ed9187baa service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:40.555466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc1e4b5-c4ab-48fc-a186-907ed9187baa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:40.564829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1e4b5-c4ab-48fc-a186-907ed9187baa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:40.566327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc1e4b5-c4ab-48fc-a186-907ed9187baa service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:40.576264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 768/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:09:42.713096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048561a1-c8af-4b06-80e7-e63f2dda24ec None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:42.713951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048561a1-c8af-4b06-80e7-e63f2dda24ec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:42.714110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048561a1-c8af-4b06-80e7-e63f2dda24ec None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:42.716396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 769/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Oct 5 14:09:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:09:42.724534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d9beb-76b3-4327-8ff0-309384c0b080 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:42.725013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d9beb-76b3-4327-8ff0-309384c0b080 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:42.725118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6d9beb-76b3-4327-8ff0-309384c0b080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:42.753465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5f6d9beb-76b3-4327-8ff0-309384c0b080 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:09:42.754727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5f6d9beb-76b3-4327-8ff0-309384c0b080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:42.763850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 769/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Oct 5 14:09:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:09:43.071754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048561a1-c8af-4b06-80e7-e63f2dda24ec None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:09:43.073838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-048561a1-c8af-4b06-80e7-e63f2dda24ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:09:43.088645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048561a1-c8af-4b06-80e7-e63f2dda24ec None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:09:43.093054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0633588-fc6c-403f-bd54-35c260a5c7c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:09:43.093676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0633588-fc6c-403f-bd54-35c260a5c7c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:09:43.093974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0633588-fc6c-403f-bd54-35c260a5c7c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:09:43.094238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0633588-fc6c-403f-bd54-35c260a5c7c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:09:43.106945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0633588-fc6c-403f-bd54-35c260a5c7c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:09:43.108372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0633588-fc6c-403f-bd54-35c260a5c7c7 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:09:43.118271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 770/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:09:52.743182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=66 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:09:53.113981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=34 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:06.032395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d846ec-3af3-4690-8780-1ed765838de9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.033054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d846ec-3af3-4690-8780-1ed765838de9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.033054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d846ec-3af3-4690-8780-1ed765838de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.058465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-22d846ec-3af3-4690-8780-1ed765838de9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.059724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-22d846ec-3af3-4690-8780-1ed765838de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.072595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 770/1540] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.081988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14baf-ef37-4891-8496-264b0114a959 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.084263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14baf-ef37-4891-8496-264b0114a959 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.084263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b14baf-ef37-4891-8496-264b0114a959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.112978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-10b14baf-ef37-4891-8496-264b0114a959 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.114235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-10b14baf-ef37-4891-8496-264b0114a959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.123006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 771/1541] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.127606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d846ec-3af3-4690-8780-1ed765838de9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:06.141232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d846ec-3af3-4690-8780-1ed765838de9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:06.151333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:06.152086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.152370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.152638 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.153367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:06.153694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:06.155329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:06.172921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 771/1542] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:06.185125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b14baf-ef37-4891-8496-264b0114a959 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:06.196254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b14baf-ef37-4891-8496-264b0114a959 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:06.207401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935ba56-81a2-444c-9518-b582bf1b6b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:06.208034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935ba56-81a2-444c-9518-b582bf1b6b55 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.208309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935ba56-81a2-444c-9518-b582bf1b6b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.208577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935ba56-81a2-444c-9518-b582bf1b6b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.209228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935ba56-81a2-444c-9518-b582bf1b6b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:06.211232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935ba56-81a2-444c-9518-b582bf1b6b55 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:06.272429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 772/1543] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:06.277157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:06.288803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198d539e-6611-4705-a98e-aac717864a6c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:06.381342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8e44e-4dad-4435-928c-ba2ad94b3b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:06.382033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8e44e-4dad-4435-928c-ba2ad94b3b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.382313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8e44e-4dad-4435-928c-ba2ad94b3b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.382778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e8e44e-4dad-4435-928c-ba2ad94b3b48 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.383486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8e44e-4dad-4435-928c-ba2ad94b3b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:06.385553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e8e44e-4dad-4435-928c-ba2ad94b3b48 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:06.419362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 772/1544] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:06.423751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935ba56-81a2-444c-9518-b582bf1b6b55 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:06.435810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935ba56-81a2-444c-9518-b582bf1b6b55 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:06.443330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2320d-6ed0-4db1-a1a1-442c789ecd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:06.443986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2320d-6ed0-4db1-a1a1-442c789ecd33 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.444266 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2320d-6ed0-4db1-a1a1-442c789ecd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.444541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2320d-6ed0-4db1-a1a1-442c789ecd33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.445227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2320d-6ed0-4db1-a1a1-442c789ecd33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:06.446893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2320d-6ed0-4db1-a1a1-442c789ecd33 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:06.458766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 773/1545] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:06.463263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8e44e-4dad-4435-928c-ba2ad94b3b48 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:06.474204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e8e44e-4dad-4435-928c-ba2ad94b3b48 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:06.483015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210629ab-07e2-46a8-93ce-1c44878b4e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:06.483647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210629ab-07e2-46a8-93ce-1c44878b4e96 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.483931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210629ab-07e2-46a8-93ce-1c44878b4e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.484201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210629ab-07e2-46a8-93ce-1c44878b4e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.484860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210629ab-07e2-46a8-93ce-1c44878b4e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:06.486542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210629ab-07e2-46a8-93ce-1c44878b4e96 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:06.500571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 773/1546] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:06.503280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2320d-6ed0-4db1-a1a1-442c789ecd33 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:06.519469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2320d-6ed0-4db1-a1a1-442c789ecd33 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:06.530717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26f56-4ab3-4561-9108-77b8480ef8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:06.531673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26f56-4ab3-4561-9108-77b8480ef8df admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.531957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26f56-4ab3-4561-9108-77b8480ef8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.532230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26f56-4ab3-4561-9108-77b8480ef8df admin admin] PATH_INFO: `/v3/projects/fe2d7cce735d41cd8c045af9b1375ba5/users/e6b5a3b83b904ed5831f2b4d2f22c780/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.557356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b26f56-4ab3-4561-9108-77b8480ef8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2d7cce735d41cd8c045af9b1375ba5, user_id=e6b5a3b83b904ed5831f2b4d2f22c780, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:06.560353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b26f56-4ab3-4561-9108-77b8480ef8df admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:06.584972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 774/1547] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:06 2022] PUT /identity/v3/projects/fe2d7cce735d41cd8c045af9b1375ba5/users/e6b5a3b83b904ed5831f2b4d2f22c780/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:06.601832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04c82d-0958-4cf6-a54f-a6e4e69c689f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.602166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04c82d-0958-4cf6-a54f-a6e4e69c689f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.602387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04c82d-0958-4cf6-a54f-a6e4e69c689f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.634748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9c04c82d-0958-4cf6-a54f-a6e4e69c689f None None] MFA Rules not processed for user `e6b5a3b83b904ed5831f2b4d2f22c780`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.635982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9c04c82d-0958-4cf6-a54f-a6e4e69c689f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.675710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aacf3-8717-422a-b283-89dc56b15f08 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.676050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aacf3-8717-422a-b283-89dc56b15f08 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.676256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aacf3-8717-422a-b283-89dc56b15f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.725269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3b7aacf3-8717-422a-b283-89dc56b15f08 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.726562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3b7aacf3-8717-422a-b283-89dc56b15f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.763640 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 774/1548] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.765129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca571c9-c26b-4bd3-a9e8-61eefa8c1ceb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.765528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca571c9-c26b-4bd3-a9e8-61eefa8c1ceb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.765843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca571c9-c26b-4bd3-a9e8-61eefa8c1ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.792897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ca571c9-c26b-4bd3-a9e8-61eefa8c1ceb None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.794145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ca571c9-c26b-4bd3-a9e8-61eefa8c1ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.800838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 775/1549] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.802322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127d76f-4378-444d-b5e4-3b990fc2fc0a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.802572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127d76f-4378-444d-b5e4-3b990fc2fc0a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.802841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127d76f-4378-444d-b5e4-3b990fc2fc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.803653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 775/1550] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.805295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eafd-8f6e-4df3-8321-5495c3d8aac7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.805663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eafd-8f6e-4df3-8321-5495c3d8aac7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.805889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a282eafd-8f6e-4df3-8321-5495c3d8aac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.830765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9127d76f-4378-444d-b5e4-3b990fc2fc0a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.831896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9127d76f-4378-444d-b5e4-3b990fc2fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.833556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a282eafd-8f6e-4df3-8321-5495c3d8aac7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.834802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a282eafd-8f6e-4df3-8321-5495c3d8aac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.839956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 776/1551] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.841545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6884b-6264-4cfb-b4dc-efff8d550797 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.842029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6884b-6264-4cfb-b4dc-efff8d550797 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.842645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6884b-6264-4cfb-b4dc-efff8d550797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.848615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 776/1552] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.850069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a3b4-c779-44d3-8a1d-88ad39dcc07c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.850441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a3b4-c779-44d3-8a1d-88ad39dcc07c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.850994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a3b4-c779-44d3-8a1d-88ad39dcc07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.867780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cbe6884b-6264-4cfb-b4dc-efff8d550797 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.869000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cbe6884b-6264-4cfb-b4dc-efff8d550797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.872131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5af9a3b4-c779-44d3-8a1d-88ad39dcc07c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.873279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5af9a3b4-c779-44d3-8a1d-88ad39dcc07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.877353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 777/1553] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.878594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448157eb-58b3-433c-968c-227bd94a2391 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.878879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448157eb-58b3-433c-968c-227bd94a2391 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.879149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448157eb-58b3-433c-968c-227bd94a2391 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.880501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 778/1554] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 14:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:11:06.881558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f406d-9fe4-4222-8a1f-1686a7f07016 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.881862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f406d-9fe4-4222-8a1f-1686a7f07016 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.882131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f406d-9fe4-4222-8a1f-1686a7f07016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.892652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 777/1555] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.897767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe79f-8e4b-40ee-99f6-cb056d618157 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.897767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe79f-8e4b-40ee-99f6-cb056d618157 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.897767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2fe79f-8e4b-40ee-99f6-cb056d618157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.908178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-298f406d-9fe4-4222-8a1f-1686a7f07016 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.909399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-298f406d-9fe4-4222-8a1f-1686a7f07016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.917384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eb2fe79f-8e4b-40ee-99f6-cb056d618157 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.917841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 779/1556] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.918788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb2fe79f-8e4b-40ee-99f6-cb056d618157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.919191 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6a5a-f2ee-4a2d-82ed-f0b8fa003190 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.919474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6a5a-f2ee-4a2d-82ed-f0b8fa003190 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.919782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b6a5a-f2ee-4a2d-82ed-f0b8fa003190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.926862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 778/1557] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.928008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f4ae-7bb1-46e6-89bf-cecc398bd5bf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.929253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f4ae-7bb1-46e6-89bf-cecc398bd5bf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.929253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae7f4ae-7bb1-46e6-89bf-cecc398bd5bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.940107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b86b6a5a-f2ee-4a2d-82ed-f0b8fa003190 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.941321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b86b6a5a-f2ee-4a2d-82ed-f0b8fa003190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.947873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7ae7f4ae-7bb1-46e6-89bf-cecc398bd5bf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.949011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7ae7f4ae-7bb1-46e6-89bf-cecc398bd5bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.949341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 780/1558] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.950707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c87406-3740-4c5f-a258-6000c51ba83f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.950989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c87406-3740-4c5f-a258-6000c51ba83f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.951269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c87406-3740-4c5f-a258-6000c51ba83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.960520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 779/1559] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.961566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16dd0-0bac-4c38-a947-0997e27600d2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.961872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16dd0-0bac-4c38-a947-0997e27600d2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.962137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b16dd0-0bac-4c38-a947-0997e27600d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.971383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-34c87406-3740-4c5f-a258-6000c51ba83f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:06.972584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34c87406-3740-4c5f-a258-6000c51ba83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:06.981726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 781/1560] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:06.982724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431cf13-69b1-4e71-9ffb-a007a311796f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:06.983009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431cf13-69b1-4e71-9ffb-a007a311796f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:06.983287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431cf13-69b1-4e71-9ffb-a007a311796f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:06.992537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:07.002222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-37b16dd0-0bac-4c38-a947-0997e27600d2 None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:07.003974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e431cf13-69b1-4e71-9ffb-a007a311796f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:07.004203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-37b16dd0-0bac-4c38-a947-0997e27600d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:07.005172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e431cf13-69b1-4e71-9ffb-a007a311796f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:07.013190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 782/1561] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:07.014564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67418594-dccf-4151-b9cc-38565c8ffbf5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.014847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67418594-dccf-4151-b9cc-38565c8ffbf5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.015118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67418594-dccf-4151-b9cc-38565c8ffbf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.034002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-67418594-dccf-4151-b9cc-38565c8ffbf5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:07.035191 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-67418594-dccf-4151-b9cc-38565c8ffbf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:07.043013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 783/1562] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:07.043718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67418594-dccf-4151-b9cc-38565c8ffbf5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.053026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67418594-dccf-4151-b9cc-38565c8ffbf5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.057750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.058350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.058625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.058892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.059598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:07.059897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.060922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:11:07.060922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Oct 05 14:11:07.060922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: Oct 05 14:11:07.061833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.073742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 784/1563] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:07.074551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.082939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 780/1564] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 14:11:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:07.083646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b16dd0-0bac-4c38-a947-0997e27600d2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.084132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e23937-1000-4795-88b5-e1d0b6761cf8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.093908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.094195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b16dd0-0bac-4c38-a947-0997e27600d2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.094523 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.094879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.095194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.095840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:07.096145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.097781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.099167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.099772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.100055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.100298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.100946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:07.101367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.103012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.110901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 785/1565] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:07.111811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.121255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4a29df-6f32-45c3-b54e-5723a8cfabae admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.121906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 781/1566] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:07.126237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.126633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.126925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.127207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.127576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.128220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:07.128613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.130237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.136299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19883cb0-1151-44a7-87c3-6e1b6ffc4f8b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.140918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30976ef1-8ef0-4587-8373-133d67f61f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.141504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976ef1-8ef0-4587-8373-133d67f61f60 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.141826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976ef1-8ef0-4587-8373-133d67f61f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.142243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 786/1567] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:07.142366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30976ef1-8ef0-4587-8373-133d67f61f60 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.142723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30976ef1-8ef0-4587-8373-133d67f61f60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:07.143020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30976ef1-8ef0-4587-8373-133d67f61f60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.144629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30976ef1-8ef0-4587-8373-133d67f61f60 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.146538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.151528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:07.164197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 782/1568] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:07.164197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a58288-2df0-452f-9e14-83227fd9ae9a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.164464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d5b7f-4260-4dc0-9abb-7b2118ee43ef admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.164546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a58288-2df0-452f-9e14-83227fd9ae9a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.164546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a58288-2df0-452f-9e14-83227fd9ae9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.164712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.164712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.164712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.164712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.165020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:07.165131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.167043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.179147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f4a58288-2df0-452f-9e14-83227fd9ae9a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:07.180646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f4a58288-2df0-452f-9e14-83227fd9ae9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:07.182343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 787/1569] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:07.183451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.195030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222fd7d6-3009-4b50-a9ec-84a8742cb52f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.196114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 783/1570] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:07.196745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a58288-2df0-452f-9e14-83227fd9ae9a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.203708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3266c9-ffe5-435c-a6c9-ef9ac1908d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.204310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3266c9-ffe5-435c-a6c9-ef9ac1908d8b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.204579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3266c9-ffe5-435c-a6c9-ef9ac1908d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.204841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3266c9-ffe5-435c-a6c9-ef9ac1908d8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.205476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3266c9-ffe5-435c-a6c9-ef9ac1908d8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.207449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3266c9-ffe5-435c-a6c9-ef9ac1908d8b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.210160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a58288-2df0-452f-9e14-83227fd9ae9a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.215779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d6dbf-7c13-4a84-8725-c4d3c87adf18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.216471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d6dbf-7c13-4a84-8725-c4d3c87adf18 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.216755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d6dbf-7c13-4a84-8725-c4d3c87adf18 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.217027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d6dbf-7c13-4a84-8725-c4d3c87adf18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.225324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d6dbf-7c13-4a84-8725-c4d3c87adf18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.227150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14d6dbf-7c13-4a84-8725-c4d3c87adf18 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.237048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 784/1571] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:07.237943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d6dbf-7c13-4a84-8725-c4d3c87adf18 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.238114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 788/1572] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.238913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3266c9-ffe5-435c-a6c9-ef9ac1908d8b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.250846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14d6dbf-7c13-4a84-8725-c4d3c87adf18 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.305732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3266c9-ffe5-435c-a6c9-ef9ac1908d8b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.306803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.307477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.307762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.308038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.308720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.309938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:11:07.309938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Oct 05 14:11:07.309938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: Oct 05 14:11:07.310828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725b59b-e038-493f-a747-445b342abc1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.311143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.311432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725b59b-e038-493f-a747-445b342abc1d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.311713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725b59b-e038-493f-a747-445b342abc1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.311967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8725b59b-e038-493f-a747-445b342abc1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.312595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725b59b-e038-493f-a747-445b342abc1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.314577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8725b59b-e038-493f-a747-445b342abc1d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.342349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 785/1573] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.343604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.348196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 789/1574] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.349558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725b59b-e038-493f-a747-445b342abc1d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.385547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:07.416848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd6f85f-a6d4-4bac-8e22-0bd3f0a29ac3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.420489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8725b59b-e038-493f-a747-445b342abc1d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.421657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed77a9-3c82-4496-bf86-3d11bbf057f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.422296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed77a9-3c82-4496-bf86-3d11bbf057f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.422681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed77a9-3c82-4496-bf86-3d11bbf057f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.423151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ed77a9-3c82-4496-bf86-3d11bbf057f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.423902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed77a9-3c82-4496-bf86-3d11bbf057f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.425624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc64d6b-6920-4d32-9efd-8c1cf5f8d061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.426058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ed77a9-3c82-4496-bf86-3d11bbf057f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.426375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc64d6b-6920-4d32-9efd-8c1cf5f8d061 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.426666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc64d6b-6920-4d32-9efd-8c1cf5f8d061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.426936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc64d6b-6920-4d32-9efd-8c1cf5f8d061 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.427591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc64d6b-6920-4d32-9efd-8c1cf5f8d061 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.429505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc64d6b-6920-4d32-9efd-8c1cf5f8d061 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.458886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 786/1575] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/projects => generated 401 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.458886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed77a9-3c82-4496-bf86-3d11bbf057f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.461694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 790/1576] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.462753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc64d6b-6920-4d32-9efd-8c1cf5f8d061 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.509140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:07.524477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ed77a9-3c82-4496-bf86-3d11bbf057f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.529186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a7ca-4480-4b25-93f0-752564c59774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.529464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc64d6b-6920-4d32-9efd-8c1cf5f8d061 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.529833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a7ca-4480-4b25-93f0-752564c59774 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.530113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a7ca-4480-4b25-93f0-752564c59774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.530518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68a7ca-4480-4b25-93f0-752564c59774 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.531163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68a7ca-4480-4b25-93f0-752564c59774 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.533093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68a7ca-4480-4b25-93f0-752564c59774 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.534060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.534670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.534945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.535213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.535869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:07.536195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.537805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.550052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 791/1577] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:07.551088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.561025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35842c96-2a28-40e7-8c47-b897cfafb3f5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.562944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 787/1578] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.563617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a7ca-4480-4b25-93f0-752564c59774 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.617325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.617876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.618154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.618433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.619079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.620107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:11:07.620107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Oct 05 14:11:07.620107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: Oct 05 14:11:07.621288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.624486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68a7ca-4480-4b25-93f0-752564c59774 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.629353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622e8c-ee27-41fe-a006-9a97570104a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.629906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622e8c-ee27-41fe-a006-9a97570104a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.630185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622e8c-ee27-41fe-a006-9a97570104a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.630460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5622e8c-ee27-41fe-a006-9a97570104a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.631103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5622e8c-ee27-41fe-a006-9a97570104a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.633015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5622e8c-ee27-41fe-a006-9a97570104a9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.646240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 792/1579] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.647269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.657573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 788/1580] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.657573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622e8c-ee27-41fe-a006-9a97570104a9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.657856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddbbe40-9c38-4f06-8a64-7f7156fe57ef admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.664397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c30e4-03d7-45c2-a55d-4809c221ef21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.664397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c30e4-03d7-45c2-a55d-4809c221ef21 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.664397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c30e4-03d7-45c2-a55d-4809c221ef21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.664397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c30e4-03d7-45c2-a55d-4809c221ef21 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.664814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c30e4-03d7-45c2-a55d-4809c221ef21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.666281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5622e8c-ee27-41fe-a006-9a97570104a9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.666785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c30e4-03d7-45c2-a55d-4809c221ef21 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.676419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a20d1cd-87e5-4111-8857-1f8c8639f8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.677027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d1cd-87e5-4111-8857-1f8c8639f8fa admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.677306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d1cd-87e5-4111-8857-1f8c8639f8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.677771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a20d1cd-87e5-4111-8857-1f8c8639f8fa admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.678234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20d1cd-87e5-4111-8857-1f8c8639f8fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.680178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a20d1cd-87e5-4111-8857-1f8c8639f8fa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.692346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 793/1581] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/users => generated 294 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.693310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c30e4-03d7-45c2-a55d-4809c221ef21 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.703459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 789/1582] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/users => generated 301 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.703693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c30e4-03d7-45c2-a55d-4809c221ef21 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.705324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fab1b6-151b-4a97-a97e-387b232bde5c None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.705607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fab1b6-151b-4a97-a97e-387b232bde5c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.705930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fab1b6-151b-4a97-a97e-387b232bde5c None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.707419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 790/1583] 10.222.0.23 () {54 vars in 912 bytes} [Wed Oct 5 14:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:11:07.707962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fab1b6-151b-4a97-a97e-387b232bde5c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.710350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f2295-0d11-4fb7-b2d0-c3a2af84a928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.710957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f2295-0d11-4fb7-b2d0-c3a2af84a928 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.711231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f2295-0d11-4fb7-b2d0-c3a2af84a928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.711499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f2295-0d11-4fb7-b2d0-c3a2af84a928 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.712129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f2295-0d11-4fb7-b2d0-c3a2af84a928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.714120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f2295-0d11-4fb7-b2d0-c3a2af84a928 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.720845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fab1b6-151b-4a97-a97e-387b232bde5c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.725867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e987a6-8603-44ed-8a6f-996f7708e335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.726651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e987a6-8603-44ed-8a6f-996f7708e335 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.726946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e987a6-8603-44ed-8a6f-996f7708e335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.727338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e987a6-8603-44ed-8a6f-996f7708e335 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.728203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e987a6-8603-44ed-8a6f-996f7708e335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.731227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e987a6-8603-44ed-8a6f-996f7708e335 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.737796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 794/1584] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.741756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f2295-0d11-4fb7-b2d0-c3a2af84a928 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.746651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:07.751672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f2295-0d11-4fb7-b2d0-c3a2af84a928 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.756286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd5d25-496c-4dcf-86bd-96a5af759983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.756894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd5d25-496c-4dcf-86bd-96a5af759983 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.757158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd5d25-496c-4dcf-86bd-96a5af759983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.757430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd5d25-496c-4dcf-86bd-96a5af759983 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.758097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd5d25-496c-4dcf-86bd-96a5af759983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.760019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd5d25-496c-4dcf-86bd-96a5af759983 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.763814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 791/1585] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.763814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e987a6-8603-44ed-8a6f-996f7708e335 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.787512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 795/1586] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/users => generated 289 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.787512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd5d25-496c-4dcf-86bd-96a5af759983 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.795328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd5d25-496c-4dcf-86bd-96a5af759983 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.822821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e987a6-8603-44ed-8a6f-996f7708e335 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.827725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699f55c-aad4-4650-9cf5-70e81059117c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.828341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699f55c-aad4-4650-9cf5-70e81059117c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.828615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699f55c-aad4-4650-9cf5-70e81059117c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.828879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e699f55c-aad4-4650-9cf5-70e81059117c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.829515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699f55c-aad4-4650-9cf5-70e81059117c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.831154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e699f55c-aad4-4650-9cf5-70e81059117c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.841083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 792/1587] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:07.841675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699f55c-aad4-4650-9cf5-70e81059117c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.850970 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125e847-80a4-43ee-9789-2d2d420dbd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.851620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125e847-80a4-43ee-9789-2d2d420dbd7e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.851748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e699f55c-aad4-4650-9cf5-70e81059117c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.851896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125e847-80a4-43ee-9789-2d2d420dbd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.852165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4125e847-80a4-43ee-9789-2d2d420dbd7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.852816 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125e847-80a4-43ee-9789-2d2d420dbd7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.854533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4125e847-80a4-43ee-9789-2d2d420dbd7e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.856609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08465e2-e4ed-46b5-8557-7cf9ef785a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.857185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08465e2-e4ed-46b5-8557-7cf9ef785a92 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.857452 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08465e2-e4ed-46b5-8557-7cf9ef785a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.857742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08465e2-e4ed-46b5-8557-7cf9ef785a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.858385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08465e2-e4ed-46b5-8557-7cf9ef785a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.859949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08465e2-e4ed-46b5-8557-7cf9ef785a92 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.864589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 796/1588] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:07.865709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125e847-80a4-43ee-9789-2d2d420dbd7e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.872915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 793/1589] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:07.872915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe8e9-63ac-4fb8-82f7-212cfb91fe28 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.872915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe8e9-63ac-4fb8-82f7-212cfb91fe28 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.872915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fe8e9-63ac-4fb8-82f7-212cfb91fe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.874872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:07.878736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4125e847-80a4-43ee-9789-2d2d420dbd7e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.885026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3a820-9703-4b17-9bc4-9d47f1ccd36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.885671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a820-9703-4b17-9bc4-9d47f1ccd36a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.885964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a820-9703-4b17-9bc4-9d47f1ccd36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.886235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a820-9703-4b17-9bc4-9d47f1ccd36a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.886887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3a820-9703-4b17-9bc4-9d47f1ccd36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.888667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3a820-9703-4b17-9bc4-9d47f1ccd36a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.892479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-980fe8e9-63ac-4fb8-82f7-212cfb91fe28 None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:07.893696 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-980fe8e9-63ac-4fb8-82f7-212cfb91fe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:07.901196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 797/1590] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:07.902042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3a820-9703-4b17-9bc4-9d47f1ccd36a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.912103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3a820-9703-4b17-9bc4-9d47f1ccd36a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.917763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4e280-f648-4588-88ae-37620a52bfdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.918373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e280-f648-4588-88ae-37620a52bfdc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.918677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e280-f648-4588-88ae-37620a52bfdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.918945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e4e280-f648-4588-88ae-37620a52bfdc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.919573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4e280-f648-4588-88ae-37620a52bfdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.921135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e4e280-f648-4588-88ae-37620a52bfdc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.931874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 798/1591] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:07.932632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4e280-f648-4588-88ae-37620a52bfdc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.941941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e4e280-f648-4588-88ae-37620a52bfdc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.946385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbba1d2-300c-4a72-84cf-5e431e96f5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.946955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbba1d2-300c-4a72-84cf-5e431e96f5fe admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.947224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbba1d2-300c-4a72-84cf-5e431e96f5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.947488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbba1d2-300c-4a72-84cf-5e431e96f5fe admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.948105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbba1d2-300c-4a72-84cf-5e431e96f5fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.950104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbba1d2-300c-4a72-84cf-5e431e96f5fe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.962249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 794/1592] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:07.963099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fe8e9-63ac-4fb8-82f7-212cfb91fe28 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.972863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fe8e9-63ac-4fb8-82f7-212cfb91fe28 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.977347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 799/1593] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:07 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:07.977347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbba1d2-300c-4a72-84cf-5e431e96f5fe admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:07.981160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66e2f7-7752-46f5-a986-759f91e071a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.981802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66e2f7-7752-46f5-a986-759f91e071a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.982130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66e2f7-7752-46f5-a986-759f91e071a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.982374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a66e2f7-7752-46f5-a986-759f91e071a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.983032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66e2f7-7752-46f5-a986-759f91e071a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.984631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a66e2f7-7752-46f5-a986-759f91e071a7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.986495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbba1d2-300c-4a72-84cf-5e431e96f5fe admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:07.991555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1578a8-7d41-474f-929e-f7c689948797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:07.992148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1578a8-7d41-474f-929e-f7c689948797 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:07.992417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1578a8-7d41-474f-929e-f7c689948797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:07.992687 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1578a8-7d41-474f-929e-f7c689948797 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:07.993319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1578a8-7d41-474f-929e-f7c689948797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:07.994804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 795/1594] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:07.994917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1578a8-7d41-474f-929e-f7c689948797 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:07.995596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66e2f7-7752-46f5-a986-759f91e071a7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.008804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a66e2f7-7752-46f5-a986-759f91e071a7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.008982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 800/1595] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:08.008982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1578a8-7d41-474f-929e-f7c689948797 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.013846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13716e9-347f-4e6a-b0d4-37450cee44c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.014450 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13716e9-347f-4e6a-b0d4-37450cee44c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.014724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13716e9-347f-4e6a-b0d4-37450cee44c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.014991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13716e9-347f-4e6a-b0d4-37450cee44c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.015631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13716e9-347f-4e6a-b0d4-37450cee44c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.015731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1578a8-7d41-474f-929e-f7c689948797 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.017336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13716e9-347f-4e6a-b0d4-37450cee44c5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.020793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1947d8-1a01-4255-9122-aaf6acd8ce8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.021395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1947d8-1a01-4255-9122-aaf6acd8ce8c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.021683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1947d8-1a01-4255-9122-aaf6acd8ce8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.021949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1947d8-1a01-4255-9122-aaf6acd8ce8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.022565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1947d8-1a01-4255-9122-aaf6acd8ce8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.024104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1947d8-1a01-4255-9122-aaf6acd8ce8c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.027782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 796/1596] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:08.028564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13716e9-347f-4e6a-b0d4-37450cee44c5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.039175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 801/1597] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:08.039175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1947d8-1a01-4255-9122-aaf6acd8ce8c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.039441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13716e9-347f-4e6a-b0d4-37450cee44c5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.043304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d569d-6455-446e-a2b9-c61e36afd0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.044043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d569d-6455-446e-a2b9-c61e36afd0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.044318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d569d-6455-446e-a2b9-c61e36afd0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.044583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d569d-6455-446e-a2b9-c61e36afd0d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.045211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d569d-6455-446e-a2b9-c61e36afd0d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.046877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d569d-6455-446e-a2b9-c61e36afd0d6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.048713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1947d8-1a01-4255-9122-aaf6acd8ce8c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.053089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0028e5-727e-410c-803a-56ed4eb775e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.053685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0028e5-727e-410c-803a-56ed4eb775e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.053957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0028e5-727e-410c-803a-56ed4eb775e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.054216 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0028e5-727e-410c-803a-56ed4eb775e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.054833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0028e5-727e-410c-803a-56ed4eb775e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.056368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0028e5-727e-410c-803a-56ed4eb775e1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.057163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 797/1598] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:08.057963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d569d-6455-446e-a2b9-c61e36afd0d6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.069554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 802/1599] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:08.070401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0028e5-727e-410c-803a-56ed4eb775e1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.079956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0028e5-727e-410c-803a-56ed4eb775e1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.084374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55df4b2-636c-4c3e-adda-094410964e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.084988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df4b2-636c-4c3e-adda-094410964e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.085198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df4b2-636c-4c3e-adda-094410964e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.085462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55df4b2-636c-4c3e-adda-094410964e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.086111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55df4b2-636c-4c3e-adda-094410964e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.087660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55df4b2-636c-4c3e-adda-094410964e17 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.097939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 803/1600] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:08.098569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55df4b2-636c-4c3e-adda-094410964e17 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.103008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.109126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55df4b2-636c-4c3e-adda-094410964e17 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.114008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ed540-09b1-4ea4-a654-66e360e9da20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.114594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ed540-09b1-4ea4-a654-66e360e9da20 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.114863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ed540-09b1-4ea4-a654-66e360e9da20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.115122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3ed540-09b1-4ea4-a654-66e360e9da20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.115746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ed540-09b1-4ea4-a654-66e360e9da20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.117304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3ed540-09b1-4ea4-a654-66e360e9da20 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.118321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d569d-6455-446e-a2b9-c61e36afd0d6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.122172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308990f-b749-44d3-a8ac-43bc75024b74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.122787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308990f-b749-44d3-a8ac-43bc75024b74 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.123115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308990f-b749-44d3-a8ac-43bc75024b74 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.123324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7308990f-b749-44d3-a8ac-43bc75024b74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.127062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 804/1601] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:08.127818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ed540-09b1-4ea4-a654-66e360e9da20 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.133817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308990f-b749-44d3-a8ac-43bc75024b74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.135252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7308990f-b749-44d3-a8ac-43bc75024b74 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.136745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3ed540-09b1-4ea4-a654-66e360e9da20 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.141283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad20ff-5011-41d5-9892-a1be0f8125ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.141939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad20ff-5011-41d5-9892-a1be0f8125ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.142212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad20ff-5011-41d5-9892-a1be0f8125ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.142476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad20ff-5011-41d5-9892-a1be0f8125ec admin admin] PATH_INFO: `/v3/projects/c42f92d724d24ef5a416f30a4c972857/users/35b4afac21e1469fac021bbb6fcf879e/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.145950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 798/1602] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:08.146712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308990f-b749-44d3-a8ac-43bc75024b74 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.156174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7308990f-b749-44d3-a8ac-43bc75024b74 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.158667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad20ff-5011-41d5-9892-a1be0f8125ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42f92d724d24ef5a416f30a4c972857, user_id=35b4afac21e1469fac021bbb6fcf879e, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.160805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.161284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad20ff-5011-41d5-9892-a1be0f8125ec admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.161476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.161746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.162011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] PATH_INFO: `/v3/projects/bd2db9ab90c54f1089c805ae9b00e064/users/a121e0fe7a054c64abdb52e36f122b8b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.178126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 805/1603] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:08 2022] PUT /identity/v3/projects/c42f92d724d24ef5a416f30a4c972857/users/35b4afac21e1469fac021bbb6fcf879e/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:08.178902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad20ff-5011-41d5-9892-a1be0f8125ec admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.192066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd2db9ab90c54f1089c805ae9b00e064, user_id=a121e0fe7a054c64abdb52e36f122b8b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.192066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:11:08.192066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Oct 05 14:11:08.192066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: Oct 05 14:11:08.193058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.209747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 799/1604] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:08 2022] PUT /identity/v3/projects/bd2db9ab90c54f1089c805ae9b00e064/users/a121e0fe7a054c64abdb52e36f122b8b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:08.210805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.257976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad20ff-5011-41d5-9892-a1be0f8125ec admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.279961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca4a123-51a5-47eb-a834-90f7c06fd1c8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.284989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675179be-025d-491b-bfc9-16c0c9427c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.285761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675179be-025d-491b-bfc9-16c0c9427c52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.286047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675179be-025d-491b-bfc9-16c0c9427c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.286321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675179be-025d-491b-bfc9-16c0c9427c52 admin admin] PATH_INFO: `/v3/projects/f3f20766dfb447eca50c103a12c5b881/users/5703963ac75f402da3c15cf32646aa0a/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.301385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675179be-025d-491b-bfc9-16c0c9427c52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3f20766dfb447eca50c103a12c5b881, user_id=5703963ac75f402da3c15cf32646aa0a, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.304304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675179be-025d-491b-bfc9-16c0c9427c52 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.310722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.318690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 800/1605] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:08 2022] PUT /identity/v3/projects/f3f20766dfb447eca50c103a12c5b881/users/5703963ac75f402da3c15cf32646aa0a/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:08.329823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba2a73-11ac-4dfb-b8c0-cdcbd5e2cb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.329823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2a73-11ac-4dfb-b8c0-cdcbd5e2cb80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.329823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2a73-11ac-4dfb-b8c0-cdcbd5e2cb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.329823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba2a73-11ac-4dfb-b8c0-cdcbd5e2cb80 admin admin] PATH_INFO: `/v3/projects/6bd3e20c75e94f3a9c17fcb3029bfdff/users/bad6116fbd5c407c9f44084fadd9531e/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.330263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675179be-025d-491b-bfc9-16c0c9427c52 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.343649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba2a73-11ac-4dfb-b8c0-cdcbd5e2cb80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd3e20c75e94f3a9c17fcb3029bfdff, user_id=bad6116fbd5c407c9f44084fadd9531e, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.346487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cba2a73-11ac-4dfb-b8c0-cdcbd5e2cb80 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.364134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 806/1606] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:08 2022] PUT /identity/v3/projects/6bd3e20c75e94f3a9c17fcb3029bfdff/users/bad6116fbd5c407c9f44084fadd9531e/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:08.366774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba2a73-11ac-4dfb-b8c0-cdcbd5e2cb80 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.394042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675179be-025d-491b-bfc9-16c0c9427c52 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.439409 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba2a73-11ac-4dfb-b8c0-cdcbd5e2cb80 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.444543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.446287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d32e8-a5d1-4887-82e0-c23e9e7e487b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.447017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d32e8-a5d1-4887-82e0-c23e9e7e487b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.447386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d32e8-a5d1-4887-82e0-c23e9e7e487b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.447774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d32e8-a5d1-4887-82e0-c23e9e7e487b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.448799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d32e8-a5d1-4887-82e0-c23e9e7e487b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.451175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d32e8-a5d1-4887-82e0-c23e9e7e487b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.459591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16db6d3-6333-44b0-9761-7127223503a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.460292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16db6d3-6333-44b0-9761-7127223503a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.460714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16db6d3-6333-44b0-9761-7127223503a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.460982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16db6d3-6333-44b0-9761-7127223503a1 admin admin] PATH_INFO: `/v3/projects/d701a6bfc69648daaaefdfeb17912d00/users/5e700616b356437c82a3a4b1d1533f6e/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.463674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 807/1607] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:08 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:08.466674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d32e8-a5d1-4887-82e0-c23e9e7e487b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.477153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d32e8-a5d1-4887-82e0-c23e9e7e487b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.477570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16db6d3-6333-44b0-9761-7127223503a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d701a6bfc69648daaaefdfeb17912d00, user_id=5e700616b356437c82a3a4b1d1533f6e, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.480607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16db6d3-6333-44b0-9761-7127223503a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.482363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ccb4e-22df-4394-a777-ff0c9fdacd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.483255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ccb4e-22df-4394-a777-ff0c9fdacd1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.483997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ccb4e-22df-4394-a777-ff0c9fdacd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.484466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ccb4e-22df-4394-a777-ff0c9fdacd1b admin admin] PATH_INFO: `/v3/projects/04395d53d99840c3b16d49e2f6d62631/users/596b52fe5595438d8acd593c61c91f67/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.496005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 801/1608] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:08 2022] PUT /identity/v3/projects/d701a6bfc69648daaaefdfeb17912d00/users/5e700616b356437c82a3a4b1d1533f6e/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:08.497226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9fcc-4714-48a1-8b71-2d905ad8a5c7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.497605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9fcc-4714-48a1-8b71-2d905ad8a5c7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.497951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ac9fcc-4714-48a1-8b71-2d905ad8a5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.512940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ccb4e-22df-4394-a777-ff0c9fdacd1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=04395d53d99840c3b16d49e2f6d62631, user_id=596b52fe5595438d8acd593c61c91f67, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.515748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ccb4e-22df-4394-a777-ff0c9fdacd1b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.532934 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 808/1609] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:08 2022] PUT /identity/v3/projects/04395d53d99840c3b16d49e2f6d62631/users/596b52fe5595438d8acd593c61c91f67/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:08.534615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f06c-4cb7-457a-a505-396278365ed4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.534978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f06c-4cb7-457a-a505-396278365ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.535248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1f06c-4cb7-457a-a505-396278365ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.539557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-35ac9fcc-4714-48a1-8b71-2d905ad8a5c7 None None] MFA Rules not processed for user `35b4afac21e1469fac021bbb6fcf879e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:08.540845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-35ac9fcc-4714-48a1-8b71-2d905ad8a5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:08.566116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-58c1f06c-4cb7-457a-a505-396278365ed4 None None] MFA Rules not processed for user `a121e0fe7a054c64abdb52e36f122b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:08.567779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-58c1f06c-4cb7-457a-a505-396278365ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:08.621679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 802/1610] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:08.621679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01270ef-02b0-48f7-8249-e3de08e30af3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.621679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01270ef-02b0-48f7-8249-e3de08e30af3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.621679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01270ef-02b0-48f7-8249-e3de08e30af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.671543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a01270ef-02b0-48f7-8249-e3de08e30af3 None None] MFA Rules not processed for user `5703963ac75f402da3c15cf32646aa0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:08.672629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a01270ef-02b0-48f7-8249-e3de08e30af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:08.679896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 809/1611] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:08.679896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80828433-5fbb-4262-96f0-adf5590e96d6 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.679896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80828433-5fbb-4262-96f0-adf5590e96d6 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.679896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80828433-5fbb-4262-96f0-adf5590e96d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.706960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-80828433-5fbb-4262-96f0-adf5590e96d6 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:08.708169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-80828433-5fbb-4262-96f0-adf5590e96d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:08.750969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.771980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 803/1612] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:08 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:08.773050 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01270ef-02b0-48f7-8249-e3de08e30af3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.814200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 810/1613] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:08.815615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d62ce-4ac3-4cf2-b38e-a01615ffae31 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.815955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d62ce-4ac3-4cf2-b38e-a01615ffae31 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.816323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276d62ce-4ac3-4cf2-b38e-a01615ffae31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.828157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:08.847359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01270ef-02b0-48f7-8249-e3de08e30af3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:08.855844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6799857-2535-404b-964c-84fce645588c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:08.856638 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6799857-2535-404b-964c-84fce645588c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.856909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6799857-2535-404b-964c-84fce645588c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.857172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6799857-2535-404b-964c-84fce645588c admin admin] PATH_INFO: `/v3/projects/45170bafd3c5447cbe6bbb69d7c5f544/users/2e988e0435464c1d8aa4842fab56dfa4/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.861568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-276d62ce-4ac3-4cf2-b38e-a01615ffae31 None None] MFA Rules not processed for user `5e700616b356437c82a3a4b1d1533f6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:08.863603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-276d62ce-4ac3-4cf2-b38e-a01615ffae31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:08.874788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6799857-2535-404b-964c-84fce645588c admin admin] RBAC: Authorizing `identity:create_grant(project_id=45170bafd3c5447cbe6bbb69d7c5f544, user_id=2e988e0435464c1d8aa4842fab56dfa4, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:08.879086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6799857-2535-404b-964c-84fce645588c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:08.899864 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 804/1614] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:08 2022] PUT /identity/v3/projects/45170bafd3c5447cbe6bbb69d7c5f544/users/2e988e0435464c1d8aa4842fab56dfa4/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:08.901529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11494ff-a2a4-4a52-9be9-13a7e472ea93 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.901834 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11494ff-a2a4-4a52-9be9-13a7e472ea93 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.902271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11494ff-a2a4-4a52-9be9-13a7e472ea93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.922212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e11494ff-a2a4-4a52-9be9-13a7e472ea93 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:08.923409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e11494ff-a2a4-4a52-9be9-13a7e472ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:08.942603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 811/1615] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:08.944075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2b816-0a5f-461d-9cf3-cc0c65b74627 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:08.944431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2b816-0a5f-461d-9cf3-cc0c65b74627 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:08.944771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c2b816-0a5f-461d-9cf3-cc0c65b74627 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:08.946202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 812/1616] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 14:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:11:08.947017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2b816-0a5f-461d-9cf3-cc0c65b74627 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.987326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 805/1617] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:08.988225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11494ff-a2a4-4a52-9be9-13a7e472ea93 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:08.999938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11494ff-a2a4-4a52-9be9-13a7e472ea93 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.004928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eb300-64f6-4687-8987-0beb1f632722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.005523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eb300-64f6-4687-8987-0beb1f632722 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.005842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eb300-64f6-4687-8987-0beb1f632722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.006089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9eb300-64f6-4687-8987-0beb1f632722 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.006199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c2b816-0a5f-461d-9cf3-cc0c65b74627 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.006717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9eb300-64f6-4687-8987-0beb1f632722 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.008625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9eb300-64f6-4687-8987-0beb1f632722 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.009906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae7c23-a177-4746-b498-9b241b4339f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.010548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae7c23-a177-4746-b498-9b241b4339f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.010820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae7c23-a177-4746-b498-9b241b4339f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.011088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae7c23-a177-4746-b498-9b241b4339f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.036692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 806/1618] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:08 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:09.037517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eb300-64f6-4687-8987-0beb1f632722 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.075148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae7c23-a177-4746-b498-9b241b4339f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.076765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae7c23-a177-4746-b498-9b241b4339f9 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.087587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 813/1619] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:08 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:09.089144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ade7f-06f1-42c1-816f-d7973e20638d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.089431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ade7f-06f1-42c1-816f-d7973e20638d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.089804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ade7f-06f1-42c1-816f-d7973e20638d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.098763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9eb300-64f6-4687-8987-0beb1f632722 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.102561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02841a-18e2-42fd-b43d-773903f8c7dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.103179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02841a-18e2-42fd-b43d-773903f8c7dc service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.103774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02841a-18e2-42fd-b43d-773903f8c7dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.104160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02841a-18e2-42fd-b43d-773903f8c7dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.111469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f63ade7f-06f1-42c1-816f-d7973e20638d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:09.113270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f63ade7f-06f1-42c1-816f-d7973e20638d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:09.131564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:09.175394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02841a-18e2-42fd-b43d-773903f8c7dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.176785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02841a-18e2-42fd-b43d-773903f8c7dc service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.181909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:09.182129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 814/1620] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:09.187202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ade7f-06f1-42c1-816f-d7973e20638d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.187818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 807/1621] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:09.189098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee764c-9266-46ec-928e-b6d1cec184cd None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.189756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee764c-9266-46ec-928e-b6d1cec184cd None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.190508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee764c-9266-46ec-928e-b6d1cec184cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.201777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ade7f-06f1-42c1-816f-d7973e20638d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.210066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90033-40f5-47f2-96c8-40ce1819350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.210750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90033-40f5-47f2-96c8-40ce1819350a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.211241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90033-40f5-47f2-96c8-40ce1819350a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.212057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d90033-40f5-47f2-96c8-40ce1819350a admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.212338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2bee764c-9266-46ec-928e-b6d1cec184cd None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:09.212855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90033-40f5-47f2-96c8-40ce1819350a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.213803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2bee764c-9266-46ec-928e-b6d1cec184cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:09.215684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d90033-40f5-47f2-96c8-40ce1819350a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.256317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 815/1622] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:09 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:09.257201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90033-40f5-47f2-96c8-40ce1819350a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.307725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 808/1623] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 14:11:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:09.318624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee764c-9266-46ec-928e-b6d1cec184cd None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.330938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee764c-9266-46ec-928e-b6d1cec184cd None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.341745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d90033-40f5-47f2-96c8-40ce1819350a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.346855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd32d13-18c3-41ff-a16f-a85bfbe19510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.347606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd32d13-18c3-41ff-a16f-a85bfbe19510 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.348187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd32d13-18c3-41ff-a16f-a85bfbe19510 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.348465 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd32d13-18c3-41ff-a16f-a85bfbe19510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.420879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aef06-9ce5-47ac-a975-704c6f06efcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.421816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aef06-9ce5-47ac-a975-704c6f06efcf admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.422315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aef06-9ce5-47ac-a975-704c6f06efcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.426014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518aef06-9ce5-47ac-a975-704c6f06efcf admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.429314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518aef06-9ce5-47ac-a975-704c6f06efcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.431472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518aef06-9ce5-47ac-a975-704c6f06efcf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.462289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd32d13-18c3-41ff-a16f-a85bfbe19510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.462474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 809/1624] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:09 2022] POST /identity/v3/users => generated 308 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:09.463642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aef06-9ce5-47ac-a975-704c6f06efcf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.467510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd32d13-18c3-41ff-a16f-a85bfbe19510 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.481311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518aef06-9ce5-47ac-a975-704c6f06efcf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.483815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 816/1625] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:09.486072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd7f07-ef79-4870-926e-edc2212f7083 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.486940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd32d13-18c3-41ff-a16f-a85bfbe19510 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.487380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7f07-ef79-4870-926e-edc2212f7083 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.487680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7f07-ef79-4870-926e-edc2212f7083 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.487916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afd7f07-ef79-4870-926e-edc2212f7083 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.498637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd7f07-ef79-4870-926e-edc2212f7083 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.500677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afd7f07-ef79-4870-926e-edc2212f7083 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.511954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd32d13-18c3-41ff-a16f-a85bfbe19510 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.520166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 810/1626] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:09.521324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd7f07-ef79-4870-926e-edc2212f7083 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.529074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7dcc26-b3f0-4f0c-acc7-eaa30132e839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.531696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dcc26-b3f0-4f0c-acc7-eaa30132e839 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.532664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dcc26-b3f0-4f0c-acc7-eaa30132e839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.533000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dcc26-b3f0-4f0c-acc7-eaa30132e839 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.534348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7dcc26-b3f0-4f0c-acc7-eaa30132e839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.537136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7dcc26-b3f0-4f0c-acc7-eaa30132e839 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.584502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 817/1627] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:09 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:09.584502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7dcc26-b3f0-4f0c-acc7-eaa30132e839 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.598993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7dcc26-b3f0-4f0c-acc7-eaa30132e839 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.606144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8d52f-b1a6-4d9b-a0df-54dba8d66f8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.607000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8d52f-b1a6-4d9b-a0df-54dba8d66f8d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.607434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8d52f-b1a6-4d9b-a0df-54dba8d66f8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.607745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c8d52f-b1a6-4d9b-a0df-54dba8d66f8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.608936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8d52f-b1a6-4d9b-a0df-54dba8d66f8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.610691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c8d52f-b1a6-4d9b-a0df-54dba8d66f8d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.620485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afd7f07-ef79-4870-926e-edc2212f7083 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.623505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 818/1628] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:09.628027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8d52f-b1a6-4d9b-a0df-54dba8d66f8d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.628353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993405a-2a06-45f0-b1aa-5fa0d01475b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.629123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993405a-2a06-45f0-b1aa-5fa0d01475b8 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.629505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993405a-2a06-45f0-b1aa-5fa0d01475b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.629832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b993405a-2a06-45f0-b1aa-5fa0d01475b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.634864 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:09.639650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c8d52f-b1a6-4d9b-a0df-54dba8d66f8d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.646207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9b6cb-155a-4624-9a6a-db57d8a24456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.646678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9b6cb-155a-4624-9a6a-db57d8a24456 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.647192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9b6cb-155a-4624-9a6a-db57d8a24456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.647522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9b6cb-155a-4624-9a6a-db57d8a24456 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.648179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9b6cb-155a-4624-9a6a-db57d8a24456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.650069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9b6cb-155a-4624-9a6a-db57d8a24456 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.681174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 819/1629] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:09.684334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9b6cb-155a-4624-9a6a-db57d8a24456 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.707733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9b6cb-155a-4624-9a6a-db57d8a24456 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.710745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993405a-2a06-45f0-b1aa-5fa0d01475b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.712529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b993405a-2a06-45f0-b1aa-5fa0d01475b8 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.723756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35ba12-1950-40ad-bae4-97c5d8bc99f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.724428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35ba12-1950-40ad-bae4-97c5d8bc99f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.724743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35ba12-1950-40ad-bae4-97c5d8bc99f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.724958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35ba12-1950-40ad-bae4-97c5d8bc99f2 admin admin] PATH_INFO: `/v3/projects/8255e083bb10467c85d8fac0016d2897/users/d1c481745e4f4dff84af33cc25673f52/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.732278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 811/1630] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:09.734942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993405a-2a06-45f0-b1aa-5fa0d01475b8 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.747669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:09.783378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35ba12-1950-40ad-bae4-97c5d8bc99f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8255e083bb10467c85d8fac0016d2897, user_id=d1c481745e4f4dff84af33cc25673f52, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.786590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35ba12-1950-40ad-bae4-97c5d8bc99f2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.792617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b993405a-2a06-45f0-b1aa-5fa0d01475b8 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:09.800676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c247-4696-4f04-9ca2-9f31092ea9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:09.803896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c247-4696-4f04-9ca2-9f31092ea9e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:09.804650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c247-4696-4f04-9ca2-9f31092ea9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:09.805197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005c247-4696-4f04-9ca2-9f31092ea9e1 admin admin] PATH_INFO: `/v3/projects/a6786adcce204e6eadad4aa8c3857144/users/5acd84d79b624a2d93ff71d7a5f29b91/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:09.818953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 820/1631] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:09 2022] PUT /identity/v3/projects/8255e083bb10467c85d8fac0016d2897/users/d1c481745e4f4dff84af33cc25673f52/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:09.823082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35ba12-1950-40ad-bae4-97c5d8bc99f2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.836549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c247-4696-4f04-9ca2-9f31092ea9e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6786adcce204e6eadad4aa8c3857144, user_id=5acd84d79b624a2d93ff71d7a5f29b91, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:09.841174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005c247-4696-4f04-9ca2-9f31092ea9e1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:09.875206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 812/1632] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:09 2022] PUT /identity/v3/projects/a6786adcce204e6eadad4aa8c3857144/users/5acd84d79b624a2d93ff71d7a5f29b91/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:09.880157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c247-4696-4f04-9ca2-9f31092ea9e1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:09.990956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35ba12-1950-40ad-bae4-97c5d8bc99f2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:10.038176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005c247-4696-4f04-9ca2-9f31092ea9e1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:10.051801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cab1ca-9a2d-4de3-8736-817b0f2d8fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:10.053157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cab1ca-9a2d-4de3-8736-817b0f2d8fb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.055203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cab1ca-9a2d-4de3-8736-817b0f2d8fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.056172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cab1ca-9a2d-4de3-8736-817b0f2d8fb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.056956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cab1ca-9a2d-4de3-8736-817b0f2d8fb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:10.060032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cab1ca-9a2d-4de3-8736-817b0f2d8fb6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:10.080996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 813/1633] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:10.086437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cab1ca-9a2d-4de3-8736-817b0f2d8fb6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:10.119252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cab1ca-9a2d-4de3-8736-817b0f2d8fb6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:10.124480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8662-de74-413f-8f02-8705f7c44e4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:10.126676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8662-de74-413f-8f02-8705f7c44e4a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.126995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8662-de74-413f-8f02-8705f7c44e4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.128719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d8662-de74-413f-8f02-8705f7c44e4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.131796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d8662-de74-413f-8f02-8705f7c44e4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:10.140269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a04515-4b02-4b97-b6b4-a97a21318c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:10.144291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a04515-4b02-4b97-b6b4-a97a21318c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.144947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a04515-4b02-4b97-b6b4-a97a21318c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.146201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a04515-4b02-4b97-b6b4-a97a21318c04 admin admin] PATH_INFO: `/v3/projects/a6786adcce204e6eadad4aa8c3857144/users/5acd84d79b624a2d93ff71d7a5f29b91/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.146524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d8662-de74-413f-8f02-8705f7c44e4a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:10.153465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a04515-4b02-4b97-b6b4-a97a21318c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6786adcce204e6eadad4aa8c3857144, user_id=5acd84d79b624a2d93ff71d7a5f29b91, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:10.173813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a04515-4b02-4b97-b6b4-a97a21318c04 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:10.185797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 821/1634] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:11:10.200571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8662-de74-413f-8f02-8705f7c44e4a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:10.242386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 814/1635] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:10 2022] PUT /identity/v3/projects/a6786adcce204e6eadad4aa8c3857144/users/5acd84d79b624a2d93ff71d7a5f29b91/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:10.242947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d8662-de74-413f-8f02-8705f7c44e4a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:10.248819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e369b-abef-44cc-a549-d283c9ac546b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.249102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e369b-abef-44cc-a549-d283c9ac546b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.249367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e369b-abef-44cc-a549-d283c9ac546b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.339516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-722e369b-abef-44cc-a549-d283c9ac546b None None] MFA Rules not processed for user `5acd84d79b624a2d93ff71d7a5f29b91`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:10.340675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-722e369b-abef-44cc-a549-d283c9ac546b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:10.411290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7882aa34-37eb-4072-bddd-a5c2cabd8ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:10.412052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882aa34-37eb-4072-bddd-a5c2cabd8ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.412806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882aa34-37eb-4072-bddd-a5c2cabd8ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.412910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7882aa34-37eb-4072-bddd-a5c2cabd8ba9 admin admin] PATH_INFO: `/v3/projects/8255e083bb10467c85d8fac0016d2897/users/d1c481745e4f4dff84af33cc25673f52/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.416770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882aa34-37eb-4072-bddd-a5c2cabd8ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8255e083bb10467c85d8fac0016d2897, user_id=d1c481745e4f4dff84af33cc25673f52, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:10.424122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7882aa34-37eb-4072-bddd-a5c2cabd8ba9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:10.461478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 822/1636] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:10 2022] PUT /identity/v3/projects/8255e083bb10467c85d8fac0016d2897/users/d1c481745e4f4dff84af33cc25673f52/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:10.461478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37139f-5294-4469-ace1-eefc277f166b None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.461478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37139f-5294-4469-ace1-eefc277f166b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.461478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37139f-5294-4469-ace1-eefc277f166b None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.461478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 823/1637] 10.222.0.23 () {54 vars in 912 bytes} [Wed Oct 5 14:11:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:11:10.463370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7426ab1-aeb4-4e6c-aa80-fce098e7ef0f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.463667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7426ab1-aeb4-4e6c-aa80-fce098e7ef0f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.463939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7426ab1-aeb4-4e6c-aa80-fce098e7ef0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.464317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:10.499107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 815/1638] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:11:10.501060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10245137-730f-462c-a357-c98f25778904 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.503446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10245137-730f-462c-a357-c98f25778904 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.503976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10245137-730f-462c-a357-c98f25778904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.525290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e7426ab1-aeb4-4e6c-aa80-fce098e7ef0f None None] MFA Rules not processed for user `d1c481745e4f4dff84af33cc25673f52`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:10.526468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e7426ab1-aeb4-4e6c-aa80-fce098e7ef0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:10.542153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-10245137-730f-462c-a357-c98f25778904 None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:10.544983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10245137-730f-462c-a357-c98f25778904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:10.552690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:10.709438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 824/1639] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:10 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:10.710478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7426ab1-aeb4-4e6c-aa80-fce098e7ef0f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:10.825583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 816/1640] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:11:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:10.869916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7426ab1-aeb4-4e6c-aa80-fce098e7ef0f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:10.881863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10245137-730f-462c-a357-c98f25778904 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:10.886054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098ce06-b906-43af-aaa9-b3f8f0402eec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:10.888345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ce06-b906-43af-aaa9-b3f8f0402eec service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.888624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ce06-b906-43af-aaa9-b3f8f0402eec service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.888894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c098ce06-b906-43af-aaa9-b3f8f0402eec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.896089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10245137-730f-462c-a357-c98f25778904 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:10.902090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375706a-720a-4aa4-b2da-57b70c444083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:10.904618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375706a-720a-4aa4-b2da-57b70c444083 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:10.904958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375706a-720a-4aa4-b2da-57b70c444083 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:10.905532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3375706a-720a-4aa4-b2da-57b70c444083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:10.926522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375706a-720a-4aa4-b2da-57b70c444083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:10.931881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3375706a-720a-4aa4-b2da-57b70c444083 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:10.960604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 817/1641] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:10 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:11.037755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098ce06-b906-43af-aaa9-b3f8f0402eec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:11.040173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c098ce06-b906-43af-aaa9-b3f8f0402eec service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:11.057827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 825/1642] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:11.119871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375706a-720a-4aa4-b2da-57b70c444083 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:11.128623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3375706a-720a-4aa4-b2da-57b70c444083 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:11.147882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af4a32-d1e7-4f52-9723-154c1f6c6208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:11.148519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af4a32-d1e7-4f52-9723-154c1f6c6208 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:11.148790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af4a32-d1e7-4f52-9723-154c1f6c6208 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:11.149054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7af4a32-d1e7-4f52-9723-154c1f6c6208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:11.168923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af4a32-d1e7-4f52-9723-154c1f6c6208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:11.172647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7af4a32-d1e7-4f52-9723-154c1f6c6208 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:11.203644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 818/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:11.642376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098ce06-b906-43af-aaa9-b3f8f0402eec service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:11.743408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af4a32-d1e7-4f52-9723-154c1f6c6208 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:11.795729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c098ce06-b906-43af-aaa9-b3f8f0402eec service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:11.824300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:11.825051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:11.825321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:11.874513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] PATH_INFO: `/v3/users/e6b5a3b83b904ed5831f2b4d2f22c780` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:11.874513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6b5a3b83b904ed5831f2b4d2f22c780)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:11.874513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:11.874513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:11.990153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7af4a32-d1e7-4f52-9723-154c1f6c6208 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:11.997785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:11.997785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:11.997785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:11.997785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] PATH_INFO: `/v3/users/5703963ac75f402da3c15cf32646aa0a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.002297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5703963ac75f402da3c15cf32646aa0a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:12.003614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: WARNING py.warnings [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:11:12.003614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: warnings.warn(msg) Oct 05 14:11:12.003614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: Oct 05 14:11:12.006973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:12.008719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6b5a3b83b904ed5831f2b4d2f22c780'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.038286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6b5a3b83b904ed5831f2b4d2f22c780'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.041259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6b5a3b83b904ed5831f2b4d2f22c780'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.060436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6b5a3b83b904ed5831f2b4d2f22c780'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.082415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6b5a3b83b904ed5831f2b4d2f22c780'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.106693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 826/1644] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:11 2022] DELETE /identity/v3/users/e6b5a3b83b904ed5831f2b4d2f22c780 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:12.107664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:12.109089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:12.144669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5703963ac75f402da3c15cf32646aa0a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.172064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5703963ac75f402da3c15cf32646aa0a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.172841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5703963ac75f402da3c15cf32646aa0a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.176377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:12.191126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5703963ac75f402da3c15cf32646aa0a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.229971 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cc75ff9e-a1c3-49ae-9e81-5863aa95e64f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5703963ac75f402da3c15cf32646aa0a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.244099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 819/1645] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:11 2022] DELETE /identity/v3/users/5703963ac75f402da3c15cf32646aa0a => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:11:12.245890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e08c2-df5b-41a3-83ca-8e271e3dd5ba None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:12.246424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e08c2-df5b-41a3-83ca-8e271e3dd5ba None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:12.246858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2e08c2-df5b-41a3-83ca-8e271e3dd5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.258219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88dd5b1-3070-4ff5-b0f7-11d538f59654 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:12.285207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9b2e08c2-df5b-41a3-83ca-8e271e3dd5ba None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:12.286716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9b2e08c2-df5b-41a3-83ca-8e271e3dd5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:12.382548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 820/1646] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:12.382548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e75c-26a7-41f3-8b25-66fa347c9bca None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:12.382548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e75c-26a7-41f3-8b25-66fa347c9bca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:12.382548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e75c-26a7-41f3-8b25-66fa347c9bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.420803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-44d9e75c-26a7-41f3-8b25-66fa347c9bca None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:12.422832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-44d9e75c-26a7-41f3-8b25-66fa347c9bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:12.436007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:12.436675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:12.436949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:12.437282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] PATH_INFO: `/v3/users/35b4afac21e1469fac021bbb6fcf879e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.441184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35b4afac21e1469fac021bbb6fcf879e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:12.443222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:12.446849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 821/1647] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:12.448086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9e75c-26a7-41f3-8b25-66fa347c9bca None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:12.450669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-44d9e75c-26a7-41f3-8b25-66fa347c9bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:12.494586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:12.564296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9e75c-26a7-41f3-8b25-66fa347c9bca None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:12.569749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:12.570887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:12.571321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:12.571728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.579789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35b4afac21e1469fac021bbb6fcf879e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.584795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:12.587211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:12.603982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 822/1648] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:12.605343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:12.606414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35b4afac21e1469fac021bbb6fcf879e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.607088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:12.609126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35b4afac21e1469fac021bbb6fcf879e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.623362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d9f265-c76b-45ea-9466-6334b11f460a service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:12.627702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35b4afac21e1469fac021bbb6fcf879e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.629931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:12.631094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:12.631541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:12.632234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] PATH_INFO: `/v3/users/5e700616b356437c82a3a4b1d1533f6e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.633775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e700616b356437c82a3a4b1d1533f6e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:12.636336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:12.642639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35b4afac21e1469fac021bbb6fcf879e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.655145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 827/1649] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:12 2022] DELETE /identity/v3/users/35b4afac21e1469fac021bbb6fcf879e => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:12.656173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:12.658573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:12.731327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e700616b356437c82a3a4b1d1533f6e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.744664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851103a5-2156-4e22-9652-53321dce1c37 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:12.751058 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb442e-30cc-4171-b240-d1f883790e76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:12.752342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb442e-30cc-4171-b240-d1f883790e76 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:12.752454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb442e-30cc-4171-b240-d1f883790e76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:12.752769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb442e-30cc-4171-b240-d1f883790e76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.755675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e700616b356437c82a3a4b1d1533f6e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.756552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e700616b356437c82a3a4b1d1533f6e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.757686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e9cb442e-30cc-4171-b240-d1f883790e76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:12.780952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e700616b356437c82a3a4b1d1533f6e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.783475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:12.806446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7f8e9530-9e07-4f3c-bd3a-32edce1b21ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e700616b356437c82a3a4b1d1533f6e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:12.818023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 823/1650] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:12 2022] DELETE /identity/v3/users/5e700616b356437c82a3a4b1d1533f6e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:12.820903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88238b-e320-4ce0-8554-42cc166eb184 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:12.821188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88238b-e320-4ce0-8554-42cc166eb184 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:12.821461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88238b-e320-4ce0-8554-42cc166eb184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.868042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb442e-30cc-4171-b240-d1f883790e76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:12.868419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2d88238b-e320-4ce0-8554-42cc166eb184 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:12.870630 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb442e-30cc-4171-b240-d1f883790e76 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:12.870820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2d88238b-e320-4ce0-8554-42cc166eb184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:12.895242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 828/1651] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:12.901689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283edf-b704-4243-8b36-4df0e26339c0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:12.902357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283edf-b704-4243-8b36-4df0e26339c0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:12.903163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39283edf-b704-4243-8b36-4df0e26339c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:12.931862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-39283edf-b704-4243-8b36-4df0e26339c0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:12.933454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39283edf-b704-4243-8b36-4df0e26339c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:13.042843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 824/1652] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:13.062710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:13.080814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 829/1653] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:13.109853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88238b-e320-4ce0-8554-42cc166eb184 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:13.112546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2d88238b-e320-4ce0-8554-42cc166eb184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:13.161210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39283edf-b704-4243-8b36-4df0e26339c0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:13.162891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39283edf-b704-4243-8b36-4df0e26339c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:13.220974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88238b-e320-4ce0-8554-42cc166eb184 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:13.234702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:13.235462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:13.235806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:13.236142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:13.258600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:13.260797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:13.283292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 825/1654] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:13.289980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39283edf-b704-4243-8b36-4df0e26339c0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:13.299679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:13.300327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:13.300658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:13.300930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:13.318759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:13.322933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:13.348539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 830/1655] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:14.445156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:14.448271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:14.464061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:14.465555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:14.477536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24499b57-a5cb-411d-8682-3541651eb31d service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:14.493245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:14.495197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcc1f1-5dd8-477f-b0e4-0d977ba09651 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:14.495474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:14.496932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:14.497774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] PATH_INFO: `/v3/projects/f3f20766dfb447eca50c103a12c5b881` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:14.503612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3f20766dfb447eca50c103a12c5b881)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:14.514842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:14.516661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:14.522902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:14.523180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:14.523466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] PATH_INFO: `/v3/projects/fe2d7cce735d41cd8c045af9b1375ba5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:14.527277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2d7cce735d41cd8c045af9b1375ba5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:14.528683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: WARNING py.warnings [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 05 14:11:14.528683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: warnings.warn(msg) Oct 05 14:11:14.528683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: Oct 05 14:11:14.547230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:14.669312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3f20766dfb447eca50c103a12c5b881', 'request_id': 'req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.687836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3f20766dfb447eca50c103a12c5b881', 'request_id': 'req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.703216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3f20766dfb447eca50c103a12c5b881', 'request_id': 'req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.712491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2d7cce735d41cd8c045af9b1375ba5', 'request_id': 'req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.722798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2d7cce735d41cd8c045af9b1375ba5', 'request_id': 'req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.733281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] The token cache is being invalidate because project f3f20766dfb447eca50c103a12c5b881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:14.734562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.735952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.741007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 826/1656] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:14 2022] DELETE /identity/v3/projects/f3f20766dfb447eca50c103a12c5b881 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:14.744663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:14.748013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:14.765349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2d7cce735d41cd8c045af9b1375ba5', 'request_id': 'req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.770528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:14.792582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] The token cache is being invalidate because project fe2d7cce735d41cd8c045af9b1375ba5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:14.792896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.795658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c7d819c0-7da0-4fbc-8b17-0a82b431fb7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:14.800117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 831/1657] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:14 2022] DELETE /identity/v3/projects/fe2d7cce735d41cd8c045af9b1375ba5 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:14.804688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9cdd-ac8d-444d-959d-f874e30b7882 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:14.804970 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9cdd-ac8d-444d-959d-f874e30b7882 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:14.805242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b9cdd-ac8d-444d-959d-f874e30b7882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:14.845729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b07b9cdd-ac8d-444d-959d-f874e30b7882 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:14.846938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b07b9cdd-ac8d-444d-959d-f874e30b7882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:14.886086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3a78a-5684-425c-b8ee-2d444f7bf10e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:15.002644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 832/1658] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:11:15.010402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e18bf0-b899-43d7-b566-1d9597c50540 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:15.010930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e18bf0-b899-43d7-b566-1d9597c50540 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:15.011386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e18bf0-b899-43d7-b566-1d9597c50540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:15.047233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:15.047905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:15.048192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:15.048510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] PATH_INFO: `/v3/users/d1c481745e4f4dff84af33cc25673f52` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:15.051732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c481745e4f4dff84af33cc25673f52)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:15.055606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:15.061392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-91e18bf0-b899-43d7-b566-1d9597c50540 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:15.062659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-91e18bf0-b899-43d7-b566-1d9597c50540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:15.092540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 833/1659] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:15.096776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e18bf0-b899-43d7-b566-1d9597c50540 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:15.119380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e18bf0-b899-43d7-b566-1d9597c50540 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:15.139306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:15.139607 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:15.141382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:15.141730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:15.150385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:15.152096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:15.158284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:15.179101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c481745e4f4dff84af33cc25673f52'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.193512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 834/1660] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:11:15.194561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:15.200337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c481745e4f4dff84af33cc25673f52'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.202679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c481745e4f4dff84af33cc25673f52'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.211533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:15.224391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5238fcd-95a7-4358-8c63-a3448bfe813b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:15.225031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c481745e4f4dff84af33cc25673f52'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.241165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:15.241842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:15.242212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:15.242418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:15.246234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:15.254045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:15.260247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c481745e4f4dff84af33cc25673f52'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.272620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 827/1661] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:14 2022] DELETE /identity/v3/users/d1c481745e4f4dff84af33cc25673f52 => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:11:15.323182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 835/1662] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:15 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:15.327779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:15.329026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:15.352401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:15.354229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:15.491902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:15.598824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfebbb3-8779-4e5f-b4e4-51c28d4227fa admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:15.605272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcfb201-7036-4d1c-bd16-d0a29008cbba admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:15.616394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:15.617071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:15.617349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:15.617621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] PATH_INFO: `/v3/projects/d701a6bfc69648daaaefdfeb17912d00` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:15.618900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d701a6bfc69648daaaefdfeb17912d00)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:15.625974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:15.633999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda484-3bac-415d-947f-02dd3b66059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:15.634486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda484-3bac-415d-947f-02dd3b66059f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:15.634812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda484-3bac-415d-947f-02dd3b66059f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:15.635015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dda484-3bac-415d-947f-02dd3b66059f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:15.635671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dda484-3bac-415d-947f-02dd3b66059f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:15.637862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dda484-3bac-415d-947f-02dd3b66059f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:15.693857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 828/1663] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:15 2022] POST /identity/v3/users => generated 283 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:15.697056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda484-3bac-415d-947f-02dd3b66059f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:15.725673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dda484-3bac-415d-947f-02dd3b66059f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:15.758427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd701a6bfc69648daaaefdfeb17912d00', 'request_id': 'req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.791196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd701a6bfc69648daaaefdfeb17912d00', 'request_id': 'req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.819319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd701a6bfc69648daaaefdfeb17912d00', 'request_id': 'req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.855692 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] The token cache is being invalidate because project d701a6bfc69648daaaefdfeb17912d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:15.856291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.856684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:15.872467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 836/1664] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:15 2022] DELETE /identity/v3/projects/d701a6bfc69648daaaefdfeb17912d00 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:15.873754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:15.875385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:15.888644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:15.888644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:15.888644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:15.888644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:15.888900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:15.890313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:15.900212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c2927-1c2f-43ec-a57b-cd2c764d7341 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:15.920982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:15.921872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:15.922151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:15.922424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] PATH_INFO: `/v3/users/5acd84d79b624a2d93ff71d7a5f29b91` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:15.923753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5acd84d79b624a2d93ff71d7a5f29b91)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:15.933202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:15.945956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 829/1665] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:15.947166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:15.952532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:15.987994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fe3439-1611-4283-8d67-bbd65fae234a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:16.003951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8bec6-370c-49f4-af30-f5b91667db98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:16.006094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8bec6-370c-49f4-af30-f5b91667db98 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:16.006094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8bec6-370c-49f4-af30-f5b91667db98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:16.006094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a8bec6-370c-49f4-af30-f5b91667db98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:16.006094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8bec6-370c-49f4-af30-f5b91667db98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:16.007647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a8bec6-370c-49f4-af30-f5b91667db98 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:16.034619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 830/1666] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:15 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:16.036773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8bec6-370c-49f4-af30-f5b91667db98 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:16.038617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5acd84d79b624a2d93ff71d7a5f29b91'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.060866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a8bec6-370c-49f4-af30-f5b91667db98 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:16.065232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5acd84d79b624a2d93ff71d7a5f29b91'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.074827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5acd84d79b624a2d93ff71d7a5f29b91'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.098537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5acd84d79b624a2d93ff71d7a5f29b91'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.100492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c582db-372c-451e-8e8a-4832a5734f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:16.101238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c582db-372c-451e-8e8a-4832a5734f20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:16.101510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c582db-372c-451e-8e8a-4832a5734f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:16.101827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c582db-372c-451e-8e8a-4832a5734f20 admin admin] PATH_INFO: `/v3/projects/742d179f3db94f228de2efa4b547510e/users/d1d1e7704cd34dfaa93b0b53659e6930/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:16.104944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:16.137703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c582db-372c-451e-8e8a-4832a5734f20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=742d179f3db94f228de2efa4b547510e, user_id=d1d1e7704cd34dfaa93b0b53659e6930, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:16.140431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c582db-372c-451e-8e8a-4832a5734f20 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:16.145190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5acd84d79b624a2d93ff71d7a5f29b91'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.165164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 837/1667] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:15 2022] DELETE /identity/v3/users/5acd84d79b624a2d93ff71d7a5f29b91 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:16.166514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:16.175634 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:16.191015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 831/1668] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:16 2022] PUT /identity/v3/projects/742d179f3db94f228de2efa4b547510e/users/d1d1e7704cd34dfaa93b0b53659e6930/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:16.202825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b6f3-4ab1-42b7-8b6e-40db75a30a71 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:16.203751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b6f3-4ab1-42b7-8b6e-40db75a30a71 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:16.203751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e0b6f3-4ab1-42b7-8b6e-40db75a30a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:16.265001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c9e0b6f3-4ab1-42b7-8b6e-40db75a30a71 None None] MFA Rules not processed for user `d1d1e7704cd34dfaa93b0b53659e6930`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:16.266463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c9e0b6f3-4ab1-42b7-8b6e-40db75a30a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:16.405994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5919698c-a8fb-45b8-8505-d78d203592c0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:16.443874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 832/1669] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:16 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:11:16.448187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0b6f3-4ab1-42b7-8b6e-40db75a30a71 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:16.449613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c9e0b6f3-4ab1-42b7-8b6e-40db75a30a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:16.487551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:16.596151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e0b6f3-4ab1-42b7-8b6e-40db75a30a71 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:16.602340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67ab1b-aba3-45f7-adbf-e06ab2e60829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:16.602491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:16.602580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67ab1b-aba3-45f7-adbf-e06ab2e60829 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:16.602870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67ab1b-aba3-45f7-adbf-e06ab2e60829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:16.603143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f67ab1b-aba3-45f7-adbf-e06ab2e60829 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:16.604167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:16.604264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67ab1b-aba3-45f7-adbf-e06ab2e60829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:16.605717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:16.606160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] PATH_INFO: `/v3/projects/c42f92d724d24ef5a416f30a4c972857` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:16.609096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c42f92d724d24ef5a416f30a4c972857)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:16.609478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f67ab1b-aba3-45f7-adbf-e06ab2e60829 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:16.615678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:16.672493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 833/1670] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:16 2022] POST /identity/v3/projects => generated 363 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:16.676336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67ab1b-aba3-45f7-adbf-e06ab2e60829 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:16.705017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c42f92d724d24ef5a416f30a4c972857', 'request_id': 'req-1c5c701f-459a-47d8-a126-acec2b38bfd9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.724296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c42f92d724d24ef5a416f30a4c972857', 'request_id': 'req-1c5c701f-459a-47d8-a126-acec2b38bfd9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.744295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c42f92d724d24ef5a416f30a4c972857', 'request_id': 'req-1c5c701f-459a-47d8-a126-acec2b38bfd9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.777382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] The token cache is being invalidate because project c42f92d724d24ef5a416f30a4c972857 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:16.777841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.779645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1c5c701f-459a-47d8-a126-acec2b38bfd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:16.786182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 838/1671] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:16 2022] DELETE /identity/v3/projects/c42f92d724d24ef5a416f30a4c972857 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:16.788899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cea3f-9f91-41c1-8ff3-f2f9433e9af8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:16.788899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cea3f-9f91-41c1-8ff3-f2f9433e9af8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:16.789213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8cea3f-9f91-41c1-8ff3-f2f9433e9af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:16.801227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f67ab1b-aba3-45f7-adbf-e06ab2e60829 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:16.860059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fc8cea3f-9f91-41c1-8ff3-f2f9433e9af8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:16.864216 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fc8cea3f-9f91-41c1-8ff3-f2f9433e9af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:16.925321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:16.958116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:16.961330 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:16.961613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:16.961926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:16.962656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:16.971625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:17.019487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 834/1672] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:16 2022] POST /identity/v3/users => generated 288 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:11:17.020332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:17.032137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:17.064019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc043b9-1afa-405b-b754-afdf784e4282 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:17.081991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee63f1f-0289-43d7-a1e7-22592c57a348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:17.082650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee63f1f-0289-43d7-a1e7-22592c57a348 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:17.082932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee63f1f-0289-43d7-a1e7-22592c57a348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:17.083204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee63f1f-0289-43d7-a1e7-22592c57a348 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:17.083914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee63f1f-0289-43d7-a1e7-22592c57a348 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:17.085655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee63f1f-0289-43d7-a1e7-22592c57a348 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:17.137043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 835/1673] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:17.137461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 839/1674] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:17.138154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee63f1f-0289-43d7-a1e7-22592c57a348 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:17.145506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47122af4-5fac-4e4c-889c-99bf8131e648 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:17.145950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47122af4-5fac-4e4c-889c-99bf8131e648 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:17.146979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47122af4-5fac-4e4c-889c-99bf8131e648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:17.194664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee63f1f-0289-43d7-a1e7-22592c57a348 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:17.208757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63549690-9ac5-4ab0-a43d-e240676fb000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:17.209895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63549690-9ac5-4ab0-a43d-e240676fb000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:17.210341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-47122af4-5fac-4e4c-889c-99bf8131e648 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:17.211484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63549690-9ac5-4ab0-a43d-e240676fb000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:17.211584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47122af4-5fac-4e4c-889c-99bf8131e648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:17.212048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63549690-9ac5-4ab0-a43d-e240676fb000 admin admin] PATH_INFO: `/v3/projects/9fe718343f7749f8a200db7425d80d2a/users/7f61a0e16c53452391f9e03467222664/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:17.241499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 840/1675] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:11:17.245594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47122af4-5fac-4e4c-889c-99bf8131e648 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:17.282623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47122af4-5fac-4e4c-889c-99bf8131e648 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:17.303519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63549690-9ac5-4ab0-a43d-e240676fb000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe718343f7749f8a200db7425d80d2a, user_id=7f61a0e16c53452391f9e03467222664, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:17.307465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63549690-9ac5-4ab0-a43d-e240676fb000 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:17.309926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:17.310704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:17.310984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:17.311250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:17.311910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:17.312205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:17.313848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:17.338514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 836/1676] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:17 2022] PUT /identity/v3/projects/9fe718343f7749f8a200db7425d80d2a/users/7f61a0e16c53452391f9e03467222664/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:17.339634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63549690-9ac5-4ab0-a43d-e240676fb000 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:17.349164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:17.367280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 841/1677] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:17.368414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:17.400959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1310ab-7f28-4fae-8260-acdc8c0093d2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:17.534475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63549690-9ac5-4ab0-a43d-e240676fb000 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:17.580572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658f9b7-758f-402f-aca4-6709135f5958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:17.586910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f9b7-758f-402f-aca4-6709135f5958 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:17.589611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f9b7-758f-402f-aca4-6709135f5958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:17.590087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f9b7-758f-402f-aca4-6709135f5958 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:17.590956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658f9b7-758f-402f-aca4-6709135f5958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:17.592920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2658f9b7-758f-402f-aca4-6709135f5958 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:17.620074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a18e71-ab1c-4dc5-89fc-c0e8980ffc92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:17.621453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a18e71-ab1c-4dc5-89fc-c0e8980ffc92 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:17.622089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a18e71-ab1c-4dc5-89fc-c0e8980ffc92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:17.622514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a18e71-ab1c-4dc5-89fc-c0e8980ffc92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:17.623410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a18e71-ab1c-4dc5-89fc-c0e8980ffc92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:17.626719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a18e71-ab1c-4dc5-89fc-c0e8980ffc92 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:17.649308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 837/1678] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:17.666617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658f9b7-758f-402f-aca4-6709135f5958 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:17.740349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 842/1679] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:17 2022] POST /identity/v3/projects => generated 381 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:11:17.745541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a18e71-ab1c-4dc5-89fc-c0e8980ffc92 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:17.766303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658f9b7-758f-402f-aca4-6709135f5958 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:17.904978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a18e71-ab1c-4dc5-89fc-c0e8980ffc92 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:17.916154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:17.916796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:17.917076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:17.917333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:17.918066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:17.920051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:17.983205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 843/1680] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:17 2022] POST /identity/v3/users => generated 291 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:17.984053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:17.987536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:18.013963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c09c0-cf95-478b-9d57-4ec003d103b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:18.014657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c09c0-cf95-478b-9d57-4ec003d103b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:18.023507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c09c0-cf95-478b-9d57-4ec003d103b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:18.023804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851c09c0-cf95-478b-9d57-4ec003d103b6 admin admin] PATH_INFO: `/v3/projects/9fe718343f7749f8a200db7425d80d2a/users/7f61a0e16c53452391f9e03467222664/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:18.037895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c09c0-cf95-478b-9d57-4ec003d103b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fe718343f7749f8a200db7425d80d2a, user_id=7f61a0e16c53452391f9e03467222664, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:18.040871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851c09c0-cf95-478b-9d57-4ec003d103b6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:18.055671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6d79d3-6da3-4847-adf9-d6e0bfdcaea3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:18.092349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:18.093073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:18.093340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:18.093605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] PATH_INFO: `/v3/projects/8255e083bb10467c85d8fac0016d2897` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:18.111158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8255e083bb10467c85d8fac0016d2897)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:18.123153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:18.145934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 838/1681] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:17 2022] PUT /identity/v3/projects/9fe718343f7749f8a200db7425d80d2a/users/7f61a0e16c53452391f9e03467222664/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:18.145934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c09c0-cf95-478b-9d57-4ec003d103b6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:18.174758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:18.269548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8255e083bb10467c85d8fac0016d2897', 'request_id': 'req-029ffb09-77a2-40fc-8a89-21d54cadf808'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.286241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8255e083bb10467c85d8fac0016d2897', 'request_id': 'req-029ffb09-77a2-40fc-8a89-21d54cadf808'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.292462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8255e083bb10467c85d8fac0016d2897', 'request_id': 'req-029ffb09-77a2-40fc-8a89-21d54cadf808'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.316087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] The token cache is being invalidate because project 8255e083bb10467c85d8fac0016d2897 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:18.323196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.325576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-029ffb09-77a2-40fc-8a89-21d54cadf808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.333929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 844/1682] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:17 2022] DELETE /identity/v3/projects/8255e083bb10467c85d8fac0016d2897 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:18.333929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110ed5c-6df4-4630-b33e-b30b8e8a890d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:18.333929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110ed5c-6df4-4630-b33e-b30b8e8a890d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:18.333929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7110ed5c-6df4-4630-b33e-b30b8e8a890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:18.338902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851c09c0-cf95-478b-9d57-4ec003d103b6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:18.423143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7110ed5c-6df4-4630-b33e-b30b8e8a890d None None] MFA Rules not processed for user `7f61a0e16c53452391f9e03467222664`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:18.431395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7110ed5c-6df4-4630-b33e-b30b8e8a890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:18.432783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:18.483047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cd37c-40a8-462d-8b90-c0972663b704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:18.484026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cd37c-40a8-462d-8b90-c0972663b704 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:18.484497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cd37c-40a8-462d-8b90-c0972663b704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:18.484947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cd37c-40a8-462d-8b90-c0972663b704 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:18.487338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cd37c-40a8-462d-8b90-c0972663b704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:18.490133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cd37c-40a8-462d-8b90-c0972663b704 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:18.512680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 839/1683] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:18.514389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b53439-1e8f-41b6-bd32-3f494a37ead3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:18.514876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b53439-1e8f-41b6-bd32-3f494a37ead3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:18.516144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b53439-1e8f-41b6-bd32-3f494a37ead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:18.560968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-94b53439-1e8f-41b6-bd32-3f494a37ead3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:18.564837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-94b53439-1e8f-41b6-bd32-3f494a37ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:18.591531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 840/1684] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:18.592699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b53439-1e8f-41b6-bd32-3f494a37ead3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:18.595249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-94b53439-1e8f-41b6-bd32-3f494a37ead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:18.613176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 845/1685] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:18 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:18.616331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110ed5c-6df4-4630-b33e-b30b8e8a890d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:18.617383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b53439-1e8f-41b6-bd32-3f494a37ead3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:18.623160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7110ed5c-6df4-4630-b33e-b30b8e8a890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:18.624665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:18.628061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:18.628698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:18.629156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] PATH_INFO: `/v3/projects/a6786adcce204e6eadad4aa8c3857144` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:18.630799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6786adcce204e6eadad4aa8c3857144)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:18.637725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:18.655105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7110ed5c-6df4-4630-b33e-b30b8e8a890d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:18.673751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647d522d-fc83-431b-baed-e5f40a8737fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:18.678315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d522d-fc83-431b-baed-e5f40a8737fb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:18.681397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d522d-fc83-431b-baed-e5f40a8737fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:18.681898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647d522d-fc83-431b-baed-e5f40a8737fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:18.685002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d522d-fc83-431b-baed-e5f40a8737fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:18.688892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647d522d-fc83-431b-baed-e5f40a8737fb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:18.746906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 846/1686] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:18.749386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35955a34-9753-499b-bbdf-2d5df6b5ec7b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:18.750945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35955a34-9753-499b-bbdf-2d5df6b5ec7b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:18.751560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35955a34-9753-499b-bbdf-2d5df6b5ec7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:18.788806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-35955a34-9753-499b-bbdf-2d5df6b5ec7b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:18.791521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6786adcce204e6eadad4aa8c3857144', 'request_id': 'req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.792279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35955a34-9753-499b-bbdf-2d5df6b5ec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:18.810675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6786adcce204e6eadad4aa8c3857144', 'request_id': 'req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.824311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6786adcce204e6eadad4aa8c3857144', 'request_id': 'req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.855886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] The token cache is being invalidate because project a6786adcce204e6eadad4aa8c3857144 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:18.856705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.857593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:18.861945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 841/1687] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:18 2022] DELETE /identity/v3/projects/a6786adcce204e6eadad4aa8c3857144 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:18.862899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:18.864330 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:18.969525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 847/1688] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:18.969525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35955a34-9753-499b-bbdf-2d5df6b5ec7b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:18.974497 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35955a34-9753-499b-bbdf-2d5df6b5ec7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:19.003149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35955a34-9753-499b-bbdf-2d5df6b5ec7b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:19.013601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591ccb5-68f1-494c-9c80-15c0d1aef3e7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:19.014218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f23cb-e8c7-4a38-a5ec-723c7cfbd226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:19.015163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f23cb-e8c7-4a38-a5ec-723c7cfbd226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.015570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f23cb-e8c7-4a38-a5ec-723c7cfbd226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.015847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f23cb-e8c7-4a38-a5ec-723c7cfbd226 admin admin] PATH_INFO: `/v3/projects/0fe5ac50e74c40fdaf2022c0856ea2a4/users/4ae5b06133f74ff5bdfefc4cebbbb89f/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.027502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:19.028164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.028440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.028708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.034680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:19.040552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:19.052418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f23cb-e8c7-4a38-a5ec-723c7cfbd226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe5ac50e74c40fdaf2022c0856ea2a4, user_id=4ae5b06133f74ff5bdfefc4cebbbb89f, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:19.055670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f23cb-e8c7-4a38-a5ec-723c7cfbd226 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:19.080756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 848/1689] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:18 2022] PUT /identity/v3/projects/0fe5ac50e74c40fdaf2022c0856ea2a4/users/4ae5b06133f74ff5bdfefc4cebbbb89f/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:19.082651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9777e-5061-4fd2-9a95-383d7afc65d5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.088452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9777e-5061-4fd2-9a95-383d7afc65d5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.089007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9777e-5061-4fd2-9a95-383d7afc65d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.124430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-07d9777e-5061-4fd2-9a95-383d7afc65d5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:19.126435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-07d9777e-5061-4fd2-9a95-383d7afc65d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:19.174116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:19.175643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:19.201321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 842/1690] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:18 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:19.202525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:19.209259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:19.234000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c843a93a-b282-4233-aac0-dabd6b86945c service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:19.252176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 849/1691] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:19.255603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ee19d-2b27-421d-92e2-b196254882ca None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.256097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ee19d-2b27-421d-92e2-b196254882ca None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.256564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ee19d-2b27-421d-92e2-b196254882ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.326827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e3901-51d1-45ae-8fc8-44235c7a30a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:19.327499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e3901-51d1-45ae-8fc8-44235c7a30a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.327823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e3901-51d1-45ae-8fc8-44235c7a30a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.328145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e3901-51d1-45ae-8fc8-44235c7a30a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.329787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e3901-51d1-45ae-8fc8-44235c7a30a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:19.330056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e3901-51d1-45ae-8fc8-44235c7a30a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:19.332282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e3901-51d1-45ae-8fc8-44235c7a30a3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:19.333319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2c9ee19d-2b27-421d-92e2-b196254882ca None None] MFA Rules not processed for user `4ae5b06133f74ff5bdfefc4cebbbb89f`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:19.337270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2c9ee19d-2b27-421d-92e2-b196254882ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:19.347114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 843/1692] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:19.348644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce4cd5-f685-4fab-9c76-39482ffc91c3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.348933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce4cd5-f685-4fab-9c76-39482ffc91c3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.349210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ce4cd5-f685-4fab-9c76-39482ffc91c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.373015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f8ce4cd5-f685-4fab-9c76-39482ffc91c3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:19.375732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f8ce4cd5-f685-4fab-9c76-39482ffc91c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:19.385862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 844/1693] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:19.387102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce4cd5-f685-4fab-9c76-39482ffc91c3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:19.400344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ce4cd5-f685-4fab-9c76-39482ffc91c3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:19.408496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413634d-6574-4e76-a5cf-ae045eb068f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:19.409507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413634d-6574-4e76-a5cf-ae045eb068f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.409975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413634d-6574-4e76-a5cf-ae045eb068f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.410689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413634d-6574-4e76-a5cf-ae045eb068f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.411542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413634d-6574-4e76-a5cf-ae045eb068f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:19.417786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413634d-6574-4e76-a5cf-ae045eb068f3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:19.504597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 845/1694] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:19 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:19.505583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413634d-6574-4e76-a5cf-ae045eb068f3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:19.632174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 850/1695] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:19.633703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ee19d-2b27-421d-92e2-b196254882ca None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:19.665057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413634d-6574-4e76-a5cf-ae045eb068f3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:19.672327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:19.673057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.673461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.673904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.674855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:19.675385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:19.677870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:19.702785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 846/1696] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:19.702785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:19.721464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8abb60-2b0f-47a7-87c7-4a7ebe457544 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:19.734800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c3fbc-f0a3-4d12-a277-c2e5ff64eaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:19.735679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c3fbc-f0a3-4d12-a277-c2e5ff64eaed admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.735965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c3fbc-f0a3-4d12-a277-c2e5ff64eaed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.736245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c3fbc-f0a3-4d12-a277-c2e5ff64eaed admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.736909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c3fbc-f0a3-4d12-a277-c2e5ff64eaed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:19.739352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c3fbc-f0a3-4d12-a277-c2e5ff64eaed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:19.786356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 847/1697] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:19 2022] POST /identity/v3/users => generated 287 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:19.786356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c3fbc-f0a3-4d12-a277-c2e5ff64eaed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:19.797617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c3fbc-f0a3-4d12-a277-c2e5ff64eaed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:19.803922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc47a0c-97db-440b-8206-dcf99e7a9604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:19.804587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc47a0c-97db-440b-8206-dcf99e7a9604 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.804968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc47a0c-97db-440b-8206-dcf99e7a9604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.805247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc47a0c-97db-440b-8206-dcf99e7a9604 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.805954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc47a0c-97db-440b-8206-dcf99e7a9604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:19.808067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc47a0c-97db-440b-8206-dcf99e7a9604 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:19.837916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ee19d-2b27-421d-92e2-b196254882ca None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:19.879361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 848/1698] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:19 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:19.882297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc47a0c-97db-440b-8206-dcf99e7a9604 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:19.932892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:19.973722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:19.974368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:19.974861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:19.975067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:19.977030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:20.002521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:20.091408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc47a0c-97db-440b-8206-dcf99e7a9604 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.098659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae40235-0b76-41e2-9f78-2d8c5956a8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.099318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae40235-0b76-41e2-9f78-2d8c5956a8fe admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.099595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae40235-0b76-41e2-9f78-2d8c5956a8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.099868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae40235-0b76-41e2-9f78-2d8c5956a8fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.100846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae40235-0b76-41e2-9f78-2d8c5956a8fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.107268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae40235-0b76-41e2-9f78-2d8c5956a8fe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.125272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.127669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.147302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 851/1699] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:20.150228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 849/1700] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:19 2022] POST /identity/v3/projects => generated 399 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:20.151481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae40235-0b76-41e2-9f78-2d8c5956a8fe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.153474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.182809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca4be0a-157d-46c3-b079-3c086ca3b033 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.245740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae40235-0b76-41e2-9f78-2d8c5956a8fe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.253726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d7c02-08f5-40bb-b7e2-1032b3cc0044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.254520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7c02-08f5-40bb-b7e2-1032b3cc0044 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.254939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7c02-08f5-40bb-b7e2-1032b3cc0044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.255392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d7c02-08f5-40bb-b7e2-1032b3cc0044 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.256167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d7c02-08f5-40bb-b7e2-1032b3cc0044 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.258159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d7c02-08f5-40bb-b7e2-1032b3cc0044 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.308882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 850/1701] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:20 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:20.317508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d78d9-9fee-4ead-9a89-9d70651fb250 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.317862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdac0b-d57b-49b4-9d14-5396a1e22104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.318504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d78d9-9fee-4ead-9a89-9d70651fb250 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.318883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372d78d9-9fee-4ead-9a89-9d70651fb250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.319766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdac0b-d57b-49b4-9d14-5396a1e22104 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.320044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdac0b-d57b-49b4-9d14-5396a1e22104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.323252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bdac0b-d57b-49b4-9d14-5396a1e22104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.323980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdac0b-d57b-49b4-9d14-5396a1e22104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.328323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bdac0b-d57b-49b4-9d14-5396a1e22104 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.368408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 852/1702] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:20.371917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdac0b-d57b-49b4-9d14-5396a1e22104 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.409828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bdac0b-d57b-49b4-9d14-5396a1e22104 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.421448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-372d78d9-9fee-4ead-9a89-9d70651fb250 None None] MFA Rules not processed for user `bad6116fbd5c407c9f44084fadd9531e`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:20.423483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-372d78d9-9fee-4ead-9a89-9d70651fb250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:20.426445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9306fe-dfe9-49fd-9758-d4ce63f69ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.428221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9306fe-dfe9-49fd-9758-d4ce63f69ad5 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.428555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9306fe-dfe9-49fd-9758-d4ce63f69ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.429492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9306fe-dfe9-49fd-9758-d4ce63f69ad5 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.430956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9306fe-dfe9-49fd-9758-d4ce63f69ad5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.438050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9306fe-dfe9-49fd-9758-d4ce63f69ad5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.504800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 853/1703] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:20 2022] POST /identity/v3/users => generated 300 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:20.504800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9306fe-dfe9-49fd-9758-d4ce63f69ad5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.545703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9306fe-dfe9-49fd-9758-d4ce63f69ad5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.563647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131006d5-891b-4d2d-92cc-f272f6bd6cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.564329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131006d5-891b-4d2d-92cc-f272f6bd6cfd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.566872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131006d5-891b-4d2d-92cc-f272f6bd6cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.567672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131006d5-891b-4d2d-92cc-f272f6bd6cfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.570440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131006d5-891b-4d2d-92cc-f272f6bd6cfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.572182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131006d5-891b-4d2d-92cc-f272f6bd6cfd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.610858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 854/1704] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:20.612676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131006d5-891b-4d2d-92cc-f272f6bd6cfd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.636110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 851/1705] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:20 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:20.637431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d78d9-9fee-4ead-9a89-9d70651fb250 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.646541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131006d5-891b-4d2d-92cc-f272f6bd6cfd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.659360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918233-13c3-4d45-98bb-d6b4969dc24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.660116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918233-13c3-4d45-98bb-d6b4969dc24b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.660551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918233-13c3-4d45-98bb-d6b4969dc24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.660922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918233-13c3-4d45-98bb-d6b4969dc24b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.661682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11918233-13c3-4d45-98bb-d6b4969dc24b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.665507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11918233-13c3-4d45-98bb-d6b4969dc24b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.671024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372d78d9-9fee-4ead-9a89-9d70651fb250 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.678819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.680200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.680475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.680887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.681554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.683759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.693460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 855/1706] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:20.694407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918233-13c3-4d45-98bb-d6b4969dc24b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.701332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 852/1707] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:20.701332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.703502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:20.721230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918233-13c3-4d45-98bb-d6b4969dc24b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.736685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af926bf8-a45c-4849-bf3d-5e7bbe27dad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.737553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af926bf8-a45c-4849-bf3d-5e7bbe27dad8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.737874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af926bf8-a45c-4849-bf3d-5e7bbe27dad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.738208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af926bf8-a45c-4849-bf3d-5e7bbe27dad8 admin admin] PATH_INFO: `/v3/projects/fadd31c039c34d9f8212007bd95bf621/users/164d22702e9d47f88bc619ee0062bca3/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:20.747029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:20.774755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af926bf8-a45c-4849-bf3d-5e7bbe27dad8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fadd31c039c34d9f8212007bd95bf621, user_id=164d22702e9d47f88bc619ee0062bca3, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:20.778976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af926bf8-a45c-4849-bf3d-5e7bbe27dad8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:20.812550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 856/1708] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:20 2022] PUT /identity/v3/projects/fadd31c039c34d9f8212007bd95bf621/users/164d22702e9d47f88bc619ee0062bca3/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:20.814393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af926bf8-a45c-4849-bf3d-5e7bbe27dad8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:20.883926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:20.904143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25845110-9e65-4fa4-a308-83db3e740271 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.969491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af926bf8-a45c-4849-bf3d-5e7bbe27dad8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:20.977430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ad303-8261-439c-a2ca-35a30575b30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:20.978564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ad303-8261-439c-a2ca-35a30575b30d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:20.978928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ad303-8261-439c-a2ca-35a30575b30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:20.984251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ad303-8261-439c-a2ca-35a30575b30d admin admin] PATH_INFO: `/v3/projects/014a5f65bff548008a4c3f51cb8bb3c9/users/80b5d2c96406427c81c2c425c8d1783b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.007269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ad303-8261-439c-a2ca-35a30575b30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=014a5f65bff548008a4c3f51cb8bb3c9, user_id=80b5d2c96406427c81c2c425c8d1783b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:21.011812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ad303-8261-439c-a2ca-35a30575b30d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:21.037154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 857/1709] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:20 2022] PUT /identity/v3/projects/014a5f65bff548008a4c3f51cb8bb3c9/users/80b5d2c96406427c81c2c425c8d1783b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:21.050016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ad303-8261-439c-a2ca-35a30575b30d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:21.061296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c243a54-699f-4731-84b0-9b080681e2c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:21.062250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c243a54-699f-4731-84b0-9b080681e2c4 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.062734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c243a54-699f-4731-84b0-9b080681e2c4 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.063193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c243a54-699f-4731-84b0-9b080681e2c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.132385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ad303-8261-439c-a2ca-35a30575b30d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:21.141867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfa744-038b-41f1-b690-c83cd9d6358b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:21.142788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfa744-038b-41f1-b690-c83cd9d6358b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.143102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c243a54-699f-4731-84b0-9b080681e2c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:21.143271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfa744-038b-41f1-b690-c83cd9d6358b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.143636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edfa744-038b-41f1-b690-c83cd9d6358b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.144394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfa744-038b-41f1-b690-c83cd9d6358b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:21.146373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edfa744-038b-41f1-b690-c83cd9d6358b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:21.148379 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c243a54-699f-4731-84b0-9b080681e2c4 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:21.164073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 858/1710] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:21.167330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfa744-038b-41f1-b690-c83cd9d6358b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:21.177290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 853/1711] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:21.179053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd29a4c-ba4f-4895-9493-289ce3f28ee4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.179478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd29a4c-ba4f-4895-9493-289ce3f28ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.179904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd29a4c-ba4f-4895-9493-289ce3f28ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.192712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edfa744-038b-41f1-b690-c83cd9d6358b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:21.206357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b9113-29cd-40bf-9876-adbe09cc226a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:21.207265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b9113-29cd-40bf-9876-adbe09cc226a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.207420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b9113-29cd-40bf-9876-adbe09cc226a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.208104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b9113-29cd-40bf-9876-adbe09cc226a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.209999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b9113-29cd-40bf-9876-adbe09cc226a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:21.223056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b9113-29cd-40bf-9876-adbe09cc226a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:21.248475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fcd29a4c-ba4f-4895-9493-289ce3f28ee4 None None] MFA Rules not processed for user `80b5d2c96406427c81c2c425c8d1783b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:21.249346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 859/1712] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:21.251765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b9113-29cd-40bf-9876-adbe09cc226a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:21.253226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fcd29a4c-ba4f-4895-9493-289ce3f28ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:21.277145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b9113-29cd-40bf-9876-adbe09cc226a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:21.292313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ac10d-60b9-4ac2-8059-686d2bc16523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:21.293690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac10d-60b9-4ac2-8059-686d2bc16523 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.293973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac10d-60b9-4ac2-8059-686d2bc16523 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.294340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ac10d-60b9-4ac2-8059-686d2bc16523 admin admin] PATH_INFO: `/v3/projects/07b623ce970f4ac3891ec87390a0513f/users/960f0555013448d4af67e9a1a6e2582a/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.332082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ac10d-60b9-4ac2-8059-686d2bc16523 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07b623ce970f4ac3891ec87390a0513f, user_id=960f0555013448d4af67e9a1a6e2582a, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:21.338580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ac10d-60b9-4ac2-8059-686d2bc16523 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:21.347254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:21.375800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 860/1713] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:21 2022] PUT /identity/v3/projects/07b623ce970f4ac3891ec87390a0513f/users/960f0555013448d4af67e9a1a6e2582a/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:21.377807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ac10d-60b9-4ac2-8059-686d2bc16523 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:21.426060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 854/1714] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:21.427909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96a8a-5e30-475c-99b5-0007e3ae22dc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.429046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96a8a-5e30-475c-99b5-0007e3ae22dc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.429422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a96a8a-5e30-475c-99b5-0007e3ae22dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.480098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ac10d-60b9-4ac2-8059-686d2bc16523 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:21.486308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e4f47-b7a2-4a5b-8f99-f580bf562c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:21.487157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e4f47-b7a2-4a5b-8f99-f580bf562c0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.487432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e4f47-b7a2-4a5b-8f99-f580bf562c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.487702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211e4f47-b7a2-4a5b-8f99-f580bf562c0f admin admin] PATH_INFO: `/v3/projects/fadd31c039c34d9f8212007bd95bf621/users/164d22702e9d47f88bc619ee0062bca3/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.489657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e4f47-b7a2-4a5b-8f99-f580bf562c0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fadd31c039c34d9f8212007bd95bf621, user_id=164d22702e9d47f88bc619ee0062bca3, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:21.492463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211e4f47-b7a2-4a5b-8f99-f580bf562c0f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:21.502135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:21.523022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 861/1715] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:21 2022] PUT /identity/v3/projects/fadd31c039c34d9f8212007bd95bf621/users/164d22702e9d47f88bc619ee0062bca3/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:21.524026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e4f47-b7a2-4a5b-8f99-f580bf562c0f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:21.538754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4a96a8a-5e30-475c-99b5-0007e3ae22dc None None] MFA Rules not processed for user `960f0555013448d4af67e9a1a6e2582a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:21.539915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4a96a8a-5e30-475c-99b5-0007e3ae22dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:21.664455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211e4f47-b7a2-4a5b-8f99-f580bf562c0f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:21.675575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ae29e-20f4-440a-963c-a94acf1c0832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:21.677451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ae29e-20f4-440a-963c-a94acf1c0832 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.677660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ae29e-20f4-440a-963c-a94acf1c0832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.679102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ae29e-20f4-440a-963c-a94acf1c0832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.680235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ae29e-20f4-440a-963c-a94acf1c0832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:21.682806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866ae29e-20f4-440a-963c-a94acf1c0832 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:21.706081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 855/1716] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:21 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:21.706081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe93215-b0a4-4451-9a3d-ba4a053da1f0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:21.706081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe93215-b0a4-4451-9a3d-ba4a053da1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:21.706406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe93215-b0a4-4451-9a3d-ba4a053da1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:21.755481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 862/1717] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:21 2022] POST /identity/v3/projects => generated 371 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:21.757833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ae29e-20f4-440a-963c-a94acf1c0832 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:21.801773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1fe93215-b0a4-4451-9a3d-ba4a053da1f0 None None] MFA Rules not processed for user `164d22702e9d47f88bc619ee0062bca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:21.803306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1fe93215-b0a4-4451-9a3d-ba4a053da1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:21.984601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866ae29e-20f4-440a-963c-a94acf1c0832 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:21.996284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:21.997507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 856/1718] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:21.998615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:22.000577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:22.001011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe93215-b0a4-4451-9a3d-ba4a053da1f0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:22.001386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:22.002190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:22.015309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:22.043327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe93215-b0a4-4451-9a3d-ba4a053da1f0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:22.053763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:22.054421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:22.054694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:22.054961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:22.057224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:22.065022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:22.088107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 863/1719] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:21 2022] POST /identity/v3/projects => generated 399 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:22.100979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:22.107154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:22.141069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 857/1720] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:21 2022] POST /identity/v3/users => generated 292 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:22.150480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:22.164930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:22.289487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:22.315323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff27eb6-6949-4c8a-8f3e-ce75bd3bc917 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:22.331417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed6d30-6c45-4f40-9995-0185c16413eb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:22.340189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de161428-4974-412e-842c-5ddab147181a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:22.340530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161428-4974-412e-842c-5ddab147181a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:22.340810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161428-4974-412e-842c-5ddab147181a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:22.341088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de161428-4974-412e-842c-5ddab147181a admin admin] PATH_INFO: `/v3/users/a121e0fe7a054c64abdb52e36f122b8b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:22.342892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de161428-4974-412e-842c-5ddab147181a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a121e0fe7a054c64abdb52e36f122b8b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:22.345412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de161428-4974-412e-842c-5ddab147181a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:22.362079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:22.432559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-de161428-4974-412e-842c-5ddab147181a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a121e0fe7a054c64abdb52e36f122b8b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:22.444738 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-de161428-4974-412e-842c-5ddab147181a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a121e0fe7a054c64abdb52e36f122b8b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:22.447734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-de161428-4974-412e-842c-5ddab147181a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a121e0fe7a054c64abdb52e36f122b8b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:22.458001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:22.459865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:22.460265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:22.465456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:22.467270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-de161428-4974-412e-842c-5ddab147181a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a121e0fe7a054c64abdb52e36f122b8b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:22.472156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:22.505253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-de161428-4974-412e-842c-5ddab147181a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a121e0fe7a054c64abdb52e36f122b8b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:22.527056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 864/1721] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:22 2022] DELETE /identity/v3/users/a121e0fe7a054c64abdb52e36f122b8b => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:22.530188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de161428-4974-412e-842c-5ddab147181a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:22.534525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-de161428-4974-412e-842c-5ddab147181a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:22.649658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:22.651712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:22.667975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 858/1722] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:22 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:22.670573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:22.672047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:22.688183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98ccd3-b53b-4c05-a237-3d219a289cce service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:22.693354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9977797a-d218-4b13-9c8a-09e0475fb727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:22.694149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977797a-d218-4b13-9c8a-09e0475fb727 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:22.694422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977797a-d218-4b13-9c8a-09e0475fb727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:22.694692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9977797a-d218-4b13-9c8a-09e0475fb727 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:22.695336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977797a-d218-4b13-9c8a-09e0475fb727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:22.697242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9977797a-d218-4b13-9c8a-09e0475fb727 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:22.700273 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de161428-4974-412e-842c-5ddab147181a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:22.715502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 859/1723] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:22.717149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f55e-1938-4ccf-9db4-e1f6527b00fe None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:22.717485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f55e-1938-4ccf-9db4-e1f6527b00fe None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:22.717823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46f55e-1938-4ccf-9db4-e1f6527b00fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:22.718765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:22.719492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:22.719763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:22.720041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:22.720690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:22.722700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:22.741000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ed46f55e-1938-4ccf-9db4-e1f6527b00fe None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:22.742270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ed46f55e-1938-4ccf-9db4-e1f6527b00fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:22.769156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 860/1724] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:22.776044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f55e-1938-4ccf-9db4-e1f6527b00fe None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:22.785262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 865/1725] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:22 2022] POST /identity/v3/users => generated 306 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:22.787992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:22.790438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:22.807639 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed46f55e-1938-4ccf-9db4-e1f6527b00fe None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:22.818525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c1272-4da0-47ba-be31-ee6844dae7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:22.819983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c1272-4da0-47ba-be31-ee6844dae7f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:22.820415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c1272-4da0-47ba-be31-ee6844dae7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:22.820683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8c1272-4da0-47ba-be31-ee6844dae7f7 admin admin] PATH_INFO: `/v3/projects/aea0d39a4b4f45a19a6b9860f9525658/users/48bf71f916ac469e8b9249feebc06530/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:22.867215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c1272-4da0-47ba-be31-ee6844dae7f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea0d39a4b4f45a19a6b9860f9525658, user_id=48bf71f916ac469e8b9249feebc06530, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:22.871175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8c1272-4da0-47ba-be31-ee6844dae7f7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:22.915314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a42be-83f4-405c-804e-4fafd471d42b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:22.921425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 861/1726] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:22 2022] PUT /identity/v3/projects/aea0d39a4b4f45a19a6b9860f9525658/users/48bf71f916ac469e8b9249feebc06530/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:22.924165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c1272-4da0-47ba-be31-ee6844dae7f7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:22.988514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:23.026238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a986a45-355b-4846-81e8-7cc5eee47147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.027052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a986a45-355b-4846-81e8-7cc5eee47147 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.027489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a986a45-355b-4846-81e8-7cc5eee47147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.027974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a986a45-355b-4846-81e8-7cc5eee47147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.047549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8c1272-4da0-47ba-be31-ee6844dae7f7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:23.051864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a986a45-355b-4846-81e8-7cc5eee47147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.053613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a986a45-355b-4846-81e8-7cc5eee47147 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.057678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f01ee-cb53-4b25-9d1e-43a2a38455db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.058527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f01ee-cb53-4b25-9d1e-43a2a38455db admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.058947 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f01ee-cb53-4b25-9d1e-43a2a38455db admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.063353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f01ee-cb53-4b25-9d1e-43a2a38455db admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.064340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f01ee-cb53-4b25-9d1e-43a2a38455db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.066515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f01ee-cb53-4b25-9d1e-43a2a38455db admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.070564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:23.079306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 866/1727] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:23.081424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a986a45-355b-4846-81e8-7cc5eee47147 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:23.094867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 862/1728] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:23.100228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f01ee-cb53-4b25-9d1e-43a2a38455db admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:23.111559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a986a45-355b-4846-81e8-7cc5eee47147 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:23.117547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f01ee-cb53-4b25-9d1e-43a2a38455db admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:23.125105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871612b-bd71-45b0-81e7-94581a60e3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.126022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871612b-bd71-45b0-81e7-94581a60e3bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.126455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871612b-bd71-45b0-81e7-94581a60e3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.126816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6871612b-bd71-45b0-81e7-94581a60e3bf admin admin] PATH_INFO: `/v3/projects/20e11a21c67640d898145f40b6df4d0e/users/aa3ba9d72366488396952e1ec8d94355/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.127858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1c328-18d5-491e-a008-dde9e85faed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.128530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c328-18d5-491e-a008-dde9e85faed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.129932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c328-18d5-491e-a008-dde9e85faed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.130285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1c328-18d5-491e-a008-dde9e85faed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.131522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1c328-18d5-491e-a008-dde9e85faed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.134191 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1c328-18d5-491e-a008-dde9e85faed4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.163703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 867/1729] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:23.171143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1c328-18d5-491e-a008-dde9e85faed4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:23.181919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871612b-bd71-45b0-81e7-94581a60e3bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e11a21c67640d898145f40b6df4d0e, user_id=aa3ba9d72366488396952e1ec8d94355, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.189867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6871612b-bd71-45b0-81e7-94581a60e3bf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.216026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1c328-18d5-491e-a008-dde9e85faed4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:23.219542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 863/1730] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:23 2022] PUT /identity/v3/projects/20e11a21c67640d898145f40b6df4d0e/users/aa3ba9d72366488396952e1ec8d94355/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:23.232349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871612b-bd71-45b0-81e7-94581a60e3bf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:23.383507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c4762-fd0c-4aae-bc4d-72dcbef3adde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.390774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c4762-fd0c-4aae-bc4d-72dcbef3adde admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.390774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c4762-fd0c-4aae-bc4d-72dcbef3adde admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.390774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211c4762-fd0c-4aae-bc4d-72dcbef3adde admin admin] PATH_INFO: `/v3/projects/aea0d39a4b4f45a19a6b9860f9525658/users/48bf71f916ac469e8b9249feebc06530/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.391002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6871612b-bd71-45b0-81e7-94581a60e3bf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:23.391084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c4762-fd0c-4aae-bc4d-72dcbef3adde admin admin] RBAC: Authorizing `identity:create_grant(project_id=aea0d39a4b4f45a19a6b9860f9525658, user_id=48bf71f916ac469e8b9249feebc06530, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.393931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211c4762-fd0c-4aae-bc4d-72dcbef3adde admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.402194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd2161-8fa7-4155-8412-3ab4acb1f54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.402809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd2161-8fa7-4155-8412-3ab4acb1f54d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.403169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd2161-8fa7-4155-8412-3ab4acb1f54d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.405731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd2161-8fa7-4155-8412-3ab4acb1f54d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.407487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd2161-8fa7-4155-8412-3ab4acb1f54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.410641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd2161-8fa7-4155-8412-3ab4acb1f54d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.428949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 868/1731] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:23 2022] PUT /identity/v3/projects/aea0d39a4b4f45a19a6b9860f9525658/users/48bf71f916ac469e8b9249feebc06530/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:23.428949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c4762-fd0c-4aae-bc4d-72dcbef3adde admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:23.433853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 864/1732] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:23 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:23.435608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93060af-136f-451a-85ec-9e6316adc7b9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.435893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93060af-136f-451a-85ec-9e6316adc7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.438701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93060af-136f-451a-85ec-9e6316adc7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.488022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b93060af-136f-451a-85ec-9e6316adc7b9 None None] MFA Rules not processed for user `48bf71f916ac469e8b9249feebc06530`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:23.490386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b93060af-136f-451a-85ec-9e6316adc7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:23.624678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211c4762-fd0c-4aae-bc4d-72dcbef3adde admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:23.638768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164d22702e9d47f88bc619ee0062bca3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fadd31c039c34d9f8212007bd95bf621', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.638967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.639340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.639809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.640636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:23.643745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.649475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.663502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 865/1733] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:23 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:23.664370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93060af-136f-451a-85ec-9e6316adc7b9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:23.698016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 869/1734] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:23.698016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:23.706658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:23.800215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:23.826037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93060af-136f-451a-85ec-9e6316adc7b9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:23.853045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c73db-a0e2-468f-96dc-730ecfc90671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.854065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06dd7de-a7f7-4d00-b5a9-8d80a4a70bda tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:23.854311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c73db-a0e2-468f-96dc-730ecfc90671 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.855007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c73db-a0e2-468f-96dc-730ecfc90671 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.857564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32c73db-a0e2-468f-96dc-730ecfc90671 admin admin] PATH_INFO: `/v3/projects/20e11a21c67640d898145f40b6df4d0e/users/aa3ba9d72366488396952e1ec8d94355/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.865372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c73db-a0e2-468f-96dc-730ecfc90671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e11a21c67640d898145f40b6df4d0e, user_id=aa3ba9d72366488396952e1ec8d94355, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.867662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a48d3-1b4e-4fa0-91a8-34e0ee8a7d16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:23.869148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a48d3-1b4e-4fa0-91a8-34e0ee8a7d16 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.869427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a48d3-1b4e-4fa0-91a8-34e0ee8a7d16 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.870254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a48d3-1b4e-4fa0-91a8-34e0ee8a7d16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:23.871725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32c73db-a0e2-468f-96dc-730ecfc90671 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.910244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a48d3-1b4e-4fa0-91a8-34e0ee8a7d16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:23.911930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3a48d3-1b4e-4fa0-91a8-34e0ee8a7d16 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:23.916256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 866/1735] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:23 2022] PUT /identity/v3/projects/20e11a21c67640d898145f40b6df4d0e/users/aa3ba9d72366488396952e1ec8d94355/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:23.917127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c73db-a0e2-468f-96dc-730ecfc90671 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:23.921462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:23.943571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 870/1736] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:23 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:23.948555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43777-4515-4ed0-a096-277ad285dbd2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:23.948555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43777-4515-4ed0-a096-277ad285dbd2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:23.948746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae43777-4515-4ed0-a096-277ad285dbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:24.041082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3ae43777-4515-4ed0-a096-277ad285dbd2 None None] MFA Rules not processed for user `aa3ba9d72366488396952e1ec8d94355`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:24.051015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3ae43777-4515-4ed0-a096-277ad285dbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:24.169845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32c73db-a0e2-468f-96dc-730ecfc90671 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:24.192095 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453aaf-da04-489b-9096-1997f6460348 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164d22702e9d47f88bc619ee0062bca3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fadd31c039c34d9f8212007bd95bf621', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:24.204238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453aaf-da04-489b-9096-1997f6460348 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:24.205983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453aaf-da04-489b-9096-1997f6460348 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:24.206723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27453aaf-da04-489b-9096-1997f6460348 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:24.210477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27453aaf-da04-489b-9096-1997f6460348 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:24.213030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27453aaf-da04-489b-9096-1997f6460348 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:24.257047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 871/1737] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:23 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:24.258398 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae43777-4515-4ed0-a096-277ad285dbd2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:24.294351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 867/1738] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:23 2022] POST /identity/v3/projects => generated 353 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:24.298476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453aaf-da04-489b-9096-1997f6460348 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:24.407119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae43777-4515-4ed0-a096-277ad285dbd2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:24.474518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27453aaf-da04-489b-9096-1997f6460348 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:24.497316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164d22702e9d47f88bc619ee0062bca3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fadd31c039c34d9f8212007bd95bf621', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:24.499455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:24.500584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:24.500796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:24.503860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:24.505405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:24.507929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:24.560039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 868/1739] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:11:24.600568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:24.605500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:24.606555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:24.607132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:24.621867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:24.764080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:24.777240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:24.778921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:24.782908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:24.784756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:24.794043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 872/1740] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:24.812135 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613db810-6474-4d80-96c6-b8d44f805645 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:24.830079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b50b1-77f4-4028-9cb3-50deb158d41c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:24.830731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b50b1-77f4-4028-9cb3-50deb158d41c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:24.831008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b50b1-77f4-4028-9cb3-50deb158d41c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:24.831281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770b50b1-77f4-4028-9cb3-50deb158d41c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:25.058396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b50b1-77f4-4028-9cb3-50deb158d41c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:25.061115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770b50b1-77f4-4028-9cb3-50deb158d41c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:25.115047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 869/1741] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:24 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:25.155107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:25.158133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b50b1-77f4-4028-9cb3-50deb158d41c service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:25.164902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:25.169605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:25.203300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770b50b1-77f4-4028-9cb3-50deb158d41c service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:25.215701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '164d22702e9d47f88bc619ee0062bca3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fadd31c039c34d9f8212007bd95bf621', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:25.216714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:25.216996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:25.217263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] PATH_INFO: `/v3/projects/37c826392b384c928739da9f1b376f50` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:25.222499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Authorizing `identity:delete_project(project_id=37c826392b384c928739da9f1b376f50)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:25.224482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:25.333223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37c826392b384c928739da9f1b376f50', 'request_id': 'req-6b9453c1-ff9b-4df6-9f61-aaafc054de98'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.343652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efc50ad-1d95-42e7-a097-1507a133ea42 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:25.366361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37c826392b384c928739da9f1b376f50', 'request_id': 'req-6b9453c1-ff9b-4df6-9f61-aaafc054de98'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.396364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37c826392b384c928739da9f1b376f50', 'request_id': 'req-6b9453c1-ff9b-4df6-9f61-aaafc054de98'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.434554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] The token cache is being invalidate because project 37c826392b384c928739da9f1b376f50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:25.436980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.438633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6b9453c1-ff9b-4df6-9f61-aaafc054de98 tempest-VolumeQuotaClassesTest-1184451667 tempest-VolumeQuotaClassesTest-1184451667-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.444641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 870/1742] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:25 2022] DELETE /identity/v3/projects/37c826392b384c928739da9f1b376f50 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:25.446272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93e91-ba46-49d4-a5d1-c0fafa1b47c7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:25.449371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93e91-ba46-49d4-a5d1-c0fafa1b47c7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:25.450080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af93e91-ba46-49d4-a5d1-c0fafa1b47c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:25.530230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0af93e91-ba46-49d4-a5d1-c0fafa1b47c7 None None] MFA Rules not processed for user `596b52fe5595438d8acd593c61c91f67`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:25.531704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0af93e91-ba46-49d4-a5d1-c0fafa1b47c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:25.534855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:25.535623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:25.535991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:25.536517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] PATH_INFO: `/v3/projects/bd2db9ab90c54f1089c805ae9b00e064` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:25.538945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd2db9ab90c54f1089c805ae9b00e064)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:25.541878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:25.656376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd2db9ab90c54f1089c805ae9b00e064', 'request_id': 'req-c9178e2e-d02e-41d0-99fb-d257d2758316'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.669691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd2db9ab90c54f1089c805ae9b00e064', 'request_id': 'req-c9178e2e-d02e-41d0-99fb-d257d2758316'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.681942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd2db9ab90c54f1089c805ae9b00e064', 'request_id': 'req-c9178e2e-d02e-41d0-99fb-d257d2758316'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.716587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] The token cache is being invalidate because project bd2db9ab90c54f1089c805ae9b00e064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:25.717283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.719252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9178e2e-d02e-41d0-99fb-d257d2758316 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:25.722034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 873/1743] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:25 2022] DELETE /identity/v3/projects/bd2db9ab90c54f1089c805ae9b00e064 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:25.723424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afc575-b80f-4529-8c93-18bcda0e9d97 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:25.723704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afc575-b80f-4529-8c93-18bcda0e9d97 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:25.723984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3afc575-b80f-4529-8c93-18bcda0e9d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:25.731999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 871/1744] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:25.735659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af93e91-ba46-49d4-a5d1-c0fafa1b47c7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:25.743584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0af93e91-ba46-49d4-a5d1-c0fafa1b47c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:25.793940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b3afc575-b80f-4529-8c93-18bcda0e9d97 None None] MFA Rules not processed for user `2e988e0435464c1d8aa4842fab56dfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:25.797247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b3afc575-b80f-4529-8c93-18bcda0e9d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:25.850388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:25.869313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af93e91-ba46-49d4-a5d1-c0fafa1b47c7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:25.932572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:25.987494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 874/1745] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:25.988984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bea4e3-047f-4abe-8d57-dc1bf25102f2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:25.989265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bea4e3-047f-4abe-8d57-dc1bf25102f2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:25.989531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bea4e3-047f-4abe-8d57-dc1bf25102f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:25.991147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:25.991791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:25.992532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:25.992804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:25.997399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.041183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-28bea4e3-047f-4abe-8d57-dc1bf25102f2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:26.042290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-28bea4e3-047f-4abe-8d57-dc1bf25102f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.166499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 875/1746] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:26.168335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc637ad-3b15-433f-b501-f61f97f42d0e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:26.168752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc637ad-3b15-433f-b501-f61f97f42d0e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:26.169141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc637ad-3b15-433f-b501-f61f97f42d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:26.193554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:26.195744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:26.211958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 872/1747] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:25 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:26.212604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:26.214453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.224664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8bc637ad-3b15-433f-b501-f61f97f42d0e None None] MFA Rules not processed for user `596b52fe5595438d8acd593c61c91f67`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:26.226141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8bc637ad-3b15-433f-b501-f61f97f42d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.338945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b078996-9d23-44ab-a3ae-f7270f93fb59 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:26.350465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7afb86-8b4f-44fd-86c6-82c1a0a83c37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:26.351798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7afb86-8b4f-44fd-86c6-82c1a0a83c37 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:26.352260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7afb86-8b4f-44fd-86c6-82c1a0a83c37 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:26.352624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7afb86-8b4f-44fd-86c6-82c1a0a83c37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:26.367122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7afb86-8b4f-44fd-86c6-82c1a0a83c37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:26.369129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7afb86-8b4f-44fd-86c6-82c1a0a83c37 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:26.402805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 873/1748] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:26.405584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310accc4-7197-4e4d-8ad2-5d828cf0b87e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:26.408140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310accc4-7197-4e4d-8ad2-5d828cf0b87e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:26.409014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310accc4-7197-4e4d-8ad2-5d828cf0b87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:26.448562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 876/1749] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:26.460785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-310accc4-7197-4e4d-8ad2-5d828cf0b87e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:26.465305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-310accc4-7197-4e4d-8ad2-5d828cf0b87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.467664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc637ad-3b15-433f-b501-f61f97f42d0e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:26.469289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8bc637ad-3b15-433f-b501-f61f97f42d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.525951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc637ad-3b15-433f-b501-f61f97f42d0e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:26.526870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 874/1750] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:26.530094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310accc4-7197-4e4d-8ad2-5d828cf0b87e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:26.551783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ff42f7-09c7-4086-8406-e3056c5fdd9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:26.553253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff42f7-09c7-4086-8406-e3056c5fdd9c service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:26.556120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff42f7-09c7-4086-8406-e3056c5fdd9c service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:26.556410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ff42f7-09c7-4086-8406-e3056c5fdd9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:26.570005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e5ff42f7-09c7-4086-8406-e3056c5fdd9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.572184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310accc4-7197-4e4d-8ad2-5d828cf0b87e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:26.591883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:26.594228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:26.594496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:26.594662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:26.599556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff42f7-09c7-4086-8406-e3056c5fdd9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:26.601233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ff42f7-09c7-4086-8406-e3056c5fdd9c service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:26.601341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:26.601663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:26.603871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:26.636611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 877/1751] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:26.641434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 875/1752] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:26.642842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b482e9b-4ad5-4d16-80fa-43c3dd5934a8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:26.643597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b482e9b-4ad5-4d16-80fa-43c3dd5934a8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:26.644875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b482e9b-4ad5-4d16-80fa-43c3dd5934a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:26.649250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:26.691712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99bdfca-4408-4478-9457-82f5f80c5df8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:26.694822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8b482e9b-4ad5-4d16-80fa-43c3dd5934a8 None None] MFA Rules not processed for user `bad6116fbd5c407c9f44084fadd9531e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:26.701417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b482e9b-4ad5-4d16-80fa-43c3dd5934a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.710306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a8615-42a9-4701-9842-fe7e35ec6d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:26.710946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a8615-42a9-4701-9842-fe7e35ec6d80 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:26.711224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a8615-42a9-4701-9842-fe7e35ec6d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:26.711493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388a8615-42a9-4701-9842-fe7e35ec6d80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:26.712847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a8615-42a9-4701-9842-fe7e35ec6d80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:26.714393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388a8615-42a9-4701-9842-fe7e35ec6d80 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:26.820298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:26.838624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 876/1753] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:26 2022] POST /identity/v3/projects => generated 375 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:26.842689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a8615-42a9-4701-9842-fe7e35ec6d80 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:26.863748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 878/1754] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:26.873510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b482e9b-4ad5-4d16-80fa-43c3dd5934a8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:26.876515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b482e9b-4ad5-4d16-80fa-43c3dd5934a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:26.895076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:26.930036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b482e9b-4ad5-4d16-80fa-43c3dd5934a8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:27.015119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388a8615-42a9-4701-9842-fe7e35ec6d80 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:27.042394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542b97d-0977-4586-ab4c-67c9e6af96f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:27.043717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542b97d-0977-4586-ab4c-67c9e6af96f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:27.044001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542b97d-0977-4586-ab4c-67c9e6af96f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:27.044272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d542b97d-0977-4586-ab4c-67c9e6af96f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:27.044939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542b97d-0977-4586-ab4c-67c9e6af96f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:27.047011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d542b97d-0977-4586-ab4c-67c9e6af96f9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:27.110461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 877/1755] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:26 2022] POST /identity/v3/users => generated 288 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:11:27.115532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542b97d-0977-4586-ab4c-67c9e6af96f9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:27.146204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d542b97d-0977-4586-ab4c-67c9e6af96f9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:27.174073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d6d7-a211-46bd-b64c-7b0d41c129a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:27.174798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d6d7-a211-46bd-b64c-7b0d41c129a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:27.175080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d6d7-a211-46bd-b64c-7b0d41c129a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:27.175353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04d6d7-a211-46bd-b64c-7b0d41c129a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:27.176018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04d6d7-a211-46bd-b64c-7b0d41c129a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:27.177676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04d6d7-a211-46bd-b64c-7b0d41c129a1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:27.200819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:27.201769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:27.202208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:27.202752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] PATH_INFO: `/v3/users/4ae5b06133f74ff5bdfefc4cebbbb89f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:27.207897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae5b06133f74ff5bdfefc4cebbbb89f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:27.210655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:27.215899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 878/1756] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:27.218265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d6d7-a211-46bd-b64c-7b0d41c129a1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:27.246774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04d6d7-a211-46bd-b64c-7b0d41c129a1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:27.267574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe05f9c-8981-412f-afe0-42f3e18e4817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:27.268301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe05f9c-8981-412f-afe0-42f3e18e4817 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:27.268531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe05f9c-8981-412f-afe0-42f3e18e4817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:27.268854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe05f9c-8981-412f-afe0-42f3e18e4817 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:27.269460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe05f9c-8981-412f-afe0-42f3e18e4817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:27.275440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe05f9c-8981-412f-afe0-42f3e18e4817 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:27.312839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 879/1757] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:27.313831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe05f9c-8981-412f-afe0-42f3e18e4817 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:27.321031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae5b06133f74ff5bdfefc4cebbbb89f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:27.324632 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe05f9c-8981-412f-afe0-42f3e18e4817 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:27.329108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae5b06133f74ff5bdfefc4cebbbb89f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:27.329959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae5b06133f74ff5bdfefc4cebbbb89f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:27.330244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833b4567-68f5-4b3c-af47-c37470750330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:27.330849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b4567-68f5-4b3c-af47-c37470750330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:27.331131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b4567-68f5-4b3c-af47-c37470750330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:27.331406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833b4567-68f5-4b3c-af47-c37470750330 admin admin] PATH_INFO: `/v3/projects/313d559efad04084bd8e6d7939fe4a72/users/a5650814bb1340fba9330c04d78fb347/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:27.343714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae5b06133f74ff5bdfefc4cebbbb89f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:27.350303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b4567-68f5-4b3c-af47-c37470750330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=313d559efad04084bd8e6d7939fe4a72, user_id=a5650814bb1340fba9330c04d78fb347, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:27.353454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833b4567-68f5-4b3c-af47-c37470750330 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:27.370388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-816e2fb9-c0a3-40f2-b757-b7acd2c99e68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae5b06133f74ff5bdfefc4cebbbb89f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:27.377605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 880/1758] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:27 2022] PUT /identity/v3/projects/313d559efad04084bd8e6d7939fe4a72/users/a5650814bb1340fba9330c04d78fb347/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:27.390768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606948b3-9c94-4076-8459-0077a22c1654 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:27.391066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606948b3-9c94-4076-8459-0077a22c1654 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:27.391359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 879/1759] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:26 2022] DELETE /identity/v3/users/4ae5b06133f74ff5bdfefc4cebbbb89f => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:11:27.391456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606948b3-9c94-4076-8459-0077a22c1654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:27.397099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84011b9-56b3-4c84-937d-4d5fd853ee10 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:27.397449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84011b9-56b3-4c84-937d-4d5fd853ee10 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:27.397883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84011b9-56b3-4c84-937d-4d5fd853ee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:27.429311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-606948b3-9c94-4076-8459-0077a22c1654 None None] MFA Rules not processed for user `a5650814bb1340fba9330c04d78fb347`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:27.430061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-606948b3-9c94-4076-8459-0077a22c1654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:27.444751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e84011b9-56b3-4c84-937d-4d5fd853ee10 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:27.451436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e84011b9-56b3-4c84-937d-4d5fd853ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:27.571014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 881/1760] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:27.572246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606948b3-9c94-4076-8459-0077a22c1654 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:27.574730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-606948b3-9c94-4076-8459-0077a22c1654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:27.687077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 880/1761] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:27.748976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84011b9-56b3-4c84-937d-4d5fd853ee10 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:27.750763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e84011b9-56b3-4c84-937d-4d5fd853ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:27.757253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606948b3-9c94-4076-8459-0077a22c1654 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:27.767911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f992d9-13fd-4687-b99b-0efec99bff2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:27.768663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f992d9-13fd-4687-b99b-0efec99bff2b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:27.769043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f992d9-13fd-4687-b99b-0efec99bff2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:27.769413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f992d9-13fd-4687-b99b-0efec99bff2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:27.770194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f992d9-13fd-4687-b99b-0efec99bff2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:27.774758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f992d9-13fd-4687-b99b-0efec99bff2b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:27.790847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:27.818096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:27.835377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 882/1762] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:27 2022] POST /identity/v3/projects => generated 375 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:27.839516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f992d9-13fd-4687-b99b-0efec99bff2b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:27.922106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84011b9-56b3-4c84-937d-4d5fd853ee10 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:28.005184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f992d9-13fd-4687-b99b-0efec99bff2b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:28.019962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252d0ac-7827-4b68-96ea-c04328216d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:28.020698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d0ac-7827-4b68-96ea-c04328216d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.021066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d0ac-7827-4b68-96ea-c04328216d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.021427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252d0ac-7827-4b68-96ea-c04328216d5f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.022332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252d0ac-7827-4b68-96ea-c04328216d5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:28.027324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252d0ac-7827-4b68-96ea-c04328216d5f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:28.036954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f44e41-2a80-4657-9344-3268a26946cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:28.038205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f44e41-2a80-4657-9344-3268a26946cc service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.039111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f44e41-2a80-4657-9344-3268a26946cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.039383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f44e41-2a80-4657-9344-3268a26946cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.064139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f44e41-2a80-4657-9344-3268a26946cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:28.069992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f44e41-2a80-4657-9344-3268a26946cc service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:28.072381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 883/1763] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:27 2022] POST /identity/v3/users => generated 294 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:28.073400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252d0ac-7827-4b68-96ea-c04328216d5f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:28.094088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252d0ac-7827-4b68-96ea-c04328216d5f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:28.094843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 881/1764] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:28.106966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97085a-4cc4-45be-8702-16502a4abacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:28.107833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97085a-4cc4-45be-8702-16502a4abacd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.108388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97085a-4cc4-45be-8702-16502a4abacd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.108754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97085a-4cc4-45be-8702-16502a4abacd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.109508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97085a-4cc4-45be-8702-16502a4abacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:28.111239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97085a-4cc4-45be-8702-16502a4abacd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:28.140694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 884/1765] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:28.141479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f44e41-2a80-4657-9344-3268a26946cc service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:28.155658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f44e41-2a80-4657-9344-3268a26946cc service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:28.169435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873077c9-4718-454e-9fc7-f5c000e33a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:28.170396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873077c9-4718-454e-9fc7-f5c000e33a7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.170986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873077c9-4718-454e-9fc7-f5c000e33a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.171344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873077c9-4718-454e-9fc7-f5c000e33a7b admin admin] PATH_INFO: `/v3/projects/89d1d241875c4b119b3ee9121c316bb6/users/414968e443074f93ae036da8ba465945/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.213070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873077c9-4718-454e-9fc7-f5c000e33a7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d1d241875c4b119b3ee9121c316bb6, user_id=414968e443074f93ae036da8ba465945, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:28.215849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873077c9-4718-454e-9fc7-f5c000e33a7b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:28.240769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 882/1766] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:28 2022] PUT /identity/v3/projects/89d1d241875c4b119b3ee9121c316bb6/users/414968e443074f93ae036da8ba465945/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:28.244749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97085a-4cc4-45be-8702-16502a4abacd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:28.263778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97085a-4cc4-45be-8702-16502a4abacd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:28.307632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874a8bd-484b-42fa-a22e-c40632d97142 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.307913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874a8bd-484b-42fa-a22e-c40632d97142 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.308501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874a8bd-484b-42fa-a22e-c40632d97142 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.311720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 883/1767] 10.222.0.23 () {54 vars in 912 bytes} [Wed Oct 5 14:11:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 05 14:11:28.323424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12264a5-7b07-47a5-94e4-e36c1fb2dd88 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.323709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12264a5-7b07-47a5-94e4-e36c1fb2dd88 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.324208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12264a5-7b07-47a5-94e4-e36c1fb2dd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.430327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c12264a5-7b07-47a5-94e4-e36c1fb2dd88 None None] MFA Rules not processed for user `6c9b940cd9e0432daa235f82becb4d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:28.431564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c12264a5-7b07-47a5-94e4-e36c1fb2dd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:28.442872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26864c02-c50d-4c41-95f8-6569a888ff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:28.443507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864c02-c50d-4c41-95f8-6569a888ff03 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.443787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864c02-c50d-4c41-95f8-6569a888ff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.444055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864c02-c50d-4c41-95f8-6569a888ff03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.444714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26864c02-c50d-4c41-95f8-6569a888ff03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:28.446379 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26864c02-c50d-4c41-95f8-6569a888ff03 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:28.474351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 885/1768] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:28.478479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26864c02-c50d-4c41-95f8-6569a888ff03 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:28.499109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26864c02-c50d-4c41-95f8-6569a888ff03 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:28.514754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0fc51-64ad-44a4-a032-b4ff459df8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:28.517553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0fc51-64ad-44a4-a032-b4ff459df8dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.518077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0fc51-64ad-44a4-a032-b4ff459df8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.518608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0fc51-64ad-44a4-a032-b4ff459df8dd admin admin] PATH_INFO: `/v3/projects/89d1d241875c4b119b3ee9121c316bb6/users/414968e443074f93ae036da8ba465945/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.522186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0fc51-64ad-44a4-a032-b4ff459df8dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d1d241875c4b119b3ee9121c316bb6, user_id=414968e443074f93ae036da8ba465945, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:28.528584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0fc51-64ad-44a4-a032-b4ff459df8dd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:28.535594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 884/1769] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:11:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:28.556060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 886/1770] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:28 2022] PUT /identity/v3/projects/89d1d241875c4b119b3ee9121c316bb6/users/414968e443074f93ae036da8ba465945/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:28.558945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12264a5-7b07-47a5-94e4-e36c1fb2dd88 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:28.569382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97c4e-a44d-45f0-99ec-9f28b8e31e3f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.572460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12264a5-7b07-47a5-94e4-e36c1fb2dd88 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:28.573476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97c4e-a44d-45f0-99ec-9f28b8e31e3f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.574335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd97c4e-a44d-45f0-99ec-9f28b8e31e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.652890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-efd97c4e-a44d-45f0-99ec-9f28b8e31e3f None None] MFA Rules not processed for user `414968e443074f93ae036da8ba465945`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:28.654457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-efd97c4e-a44d-45f0-99ec-9f28b8e31e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:28.680664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:28.688897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858147-0cda-4315-a3e7-dfa08c22a64d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:28.689545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858147-0cda-4315-a3e7-dfa08c22a64d service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.691305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858147-0cda-4315-a3e7-dfa08c22a64d service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.691608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858147-0cda-4315-a3e7-dfa08c22a64d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.715834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858147-0cda-4315-a3e7-dfa08c22a64d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:28.718571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858147-0cda-4315-a3e7-dfa08c22a64d service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:28.737969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:28.751198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 885/1771] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:28.756548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d51bd-b29b-414e-b6cf-3c1f5c4dc6e0 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.756867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d51bd-b29b-414e-b6cf-3c1f5c4dc6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.757097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73d51bd-b29b-414e-b6cf-3c1f5c4dc6e0 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.759556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 886/1772] 10.222.0.23 () {54 vars in 912 bytes} [Wed Oct 5 14:11:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:11:28.760348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d51bd-b29b-414e-b6cf-3c1f5c4dc6e0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:28.765425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a73d51bd-b29b-414e-b6cf-3c1f5c4dc6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:28.808646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 887/1773] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:28.808646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6c898-81fd-45a0-bd1d-a2993b41f0fb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.808646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6c898-81fd-45a0-bd1d-a2993b41f0fb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.809590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c6c898-81fd-45a0-bd1d-a2993b41f0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.855127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-19c6c898-81fd-45a0-bd1d-a2993b41f0fb None None] MFA Rules not processed for user `6c9b940cd9e0432daa235f82becb4d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:28.862725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-19c6c898-81fd-45a0-bd1d-a2993b41f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:28.878490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 888/1774] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:11:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:28.879438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6c898-81fd-45a0-bd1d-a2993b41f0fb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:28.887481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-19c6c898-81fd-45a0-bd1d-a2993b41f0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:28.895244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73d51bd-b29b-414e-b6cf-3c1f5c4dc6e0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:28.901285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:28.902214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:28.902437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:28.902710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] PATH_INFO: `/v3/projects/0fe5ac50e74c40fdaf2022c0856ea2a4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:28.904258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe5ac50e74c40fdaf2022c0856ea2a4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:28.907088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:28.991374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c6c898-81fd-45a0-bd1d-a2993b41f0fb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:29.002522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beee18-dafc-45db-8d9e-1d0f9b73f66f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:29.003240 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beee18-dafc-45db-8d9e-1d0f9b73f66f service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:29.003572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beee18-dafc-45db-8d9e-1d0f9b73f66f service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:29.004534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05beee18-dafc-45db-8d9e-1d0f9b73f66f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:29.036162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe5ac50e74c40fdaf2022c0856ea2a4', 'request_id': 'req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.043679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe5ac50e74c40fdaf2022c0856ea2a4', 'request_id': 'req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.051964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe5ac50e74c40fdaf2022c0856ea2a4', 'request_id': 'req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.081856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] The token cache is being invalidate because project 0fe5ac50e74c40fdaf2022c0856ea2a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:29.082448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.083431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.086116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 887/1775] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:28 2022] DELETE /identity/v3/projects/0fe5ac50e74c40fdaf2022c0856ea2a4 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:29.088890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:29.094074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:29.152909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05beee18-dafc-45db-8d9e-1d0f9b73f66f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:29.154845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05beee18-dafc-45db-8d9e-1d0f9b73f66f service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:29.170988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891fc0e9-2cd2-4bd4-a405-09f097f78ea4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:29.175523 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:29.176395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:29.176837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:29.177386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:29.190581 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:29.194416 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 889/1776] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:29.197169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beee18-dafc-45db-8d9e-1d0f9b73f66f service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:29.206469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:29.208537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:29.231049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05beee18-dafc-45db-8d9e-1d0f9b73f66f service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:29.245079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 888/1777] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:29.414346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:29.415041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:29.415317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:29.417609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] PATH_INFO: `/v3/users/164d22702e9d47f88bc619ee0062bca3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:29.420758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] RBAC: Authorizing `identity:delete_user(user_id=164d22702e9d47f88bc619ee0062bca3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:29.422866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:29.516812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:29.554314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:29.555986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:29.608705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '164d22702e9d47f88bc619ee0062bca3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.631480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '164d22702e9d47f88bc619ee0062bca3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.634863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '164d22702e9d47f88bc619ee0062bca3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.653865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '164d22702e9d47f88bc619ee0062bca3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.672810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:29.695073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '164d22702e9d47f88bc619ee0062bca3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:29.713590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 890/1778] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:29 2022] DELETE /identity/v3/users/164d22702e9d47f88bc619ee0062bca3 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:29.815422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c78d1c1-9c3f-4bd9-927d-9ef2c5e958e5 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:29.981953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc3b8d-38c5-4172-8c05-991c9421ffcf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:29.981953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3b8d-38c5-4172-8c05-991c9421ffcf service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:29.981953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3b8d-38c5-4172-8c05-991c9421ffcf service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:29.981953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc3b8d-38c5-4172-8c05-991c9421ffcf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:29.989282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-54bc3b8d-38c5-4172-8c05-991c9421ffcf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:30.030018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3b8d-38c5-4172-8c05-991c9421ffcf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:30.030018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc3b8d-38c5-4172-8c05-991c9421ffcf service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:30.042724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 889/1779] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 489 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:30.322779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:30.332241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:30.333972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44436a0c-9a14-47da-9e67-99735ec6f3ec None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:30.334128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44436a0c-9a14-47da-9e67-99735ec6f3ec None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:30.334412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44436a0c-9a14-47da-9e67-99735ec6f3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:30.383293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-44436a0c-9a14-47da-9e67-99735ec6f3ec None None] MFA Rules not processed for user `2e988e0435464c1d8aa4842fab56dfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:30.384923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-44436a0c-9a14-47da-9e67-99735ec6f3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:30.507702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 890/1780] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:30.544339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3f77e1-cd7e-429f-9641-e74d0ccb6757 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:30.549615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:30.550551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:30.550982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:30.551416 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] PATH_INFO: `/v3/projects/fadd31c039c34d9f8212007bd95bf621` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:30.552788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fadd31c039c34d9f8212007bd95bf621)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:30.555128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:30.654319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fadd31c039c34d9f8212007bd95bf621', 'request_id': 'req-38521e5b-bd41-49e7-8695-3bc2fda35bf1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:30.666907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fadd31c039c34d9f8212007bd95bf621', 'request_id': 'req-38521e5b-bd41-49e7-8695-3bc2fda35bf1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:30.677441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fadd31c039c34d9f8212007bd95bf621', 'request_id': 'req-38521e5b-bd41-49e7-8695-3bc2fda35bf1'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:30.701065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] The token cache is being invalidate because project fadd31c039c34d9f8212007bd95bf621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:30.702212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:30.706065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38521e5b-bd41-49e7-8695-3bc2fda35bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:30.708883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 891/1781] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:30 2022] DELETE /identity/v3/projects/fadd31c039c34d9f8212007bd95bf621 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:30.714403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ee8be-dce5-4331-8b6a-76e50e71da74 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:30.714684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ee8be-dce5-4331-8b6a-76e50e71da74 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:30.714961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ee8be-dce5-4331-8b6a-76e50e71da74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:30.793627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1b8ee8be-dce5-4331-8b6a-76e50e71da74 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:30.794925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1b8ee8be-dce5-4331-8b6a-76e50e71da74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:30.969943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 891/1782] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:30.980048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfafa6-e0d1-4b48-80e7-7912023b6152 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:30.981083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfafa6-e0d1-4b48-80e7-7912023b6152 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:30.981508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccfafa6-e0d1-4b48-80e7-7912023b6152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:31.033094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bccfafa6-e0d1-4b48-80e7-7912023b6152 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:31.038729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bccfafa6-e0d1-4b48-80e7-7912023b6152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:31.058923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 892/1783] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:31.075919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ee8be-dce5-4331-8b6a-76e50e71da74 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:31.075919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:31.077958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ee8be-dce5-4331-8b6a-76e50e71da74 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:31.084988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:31.085730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:31.086081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:31.086416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:31.087156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:31.087529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:31.089268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:31.126094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 892/1784] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:31.127141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfafa6-e0d1-4b48-80e7-7912023b6152 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:31.155346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccfafa6-e0d1-4b48-80e7-7912023b6152 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:31.167869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da2335-81f4-4f7b-8103-4b52df115d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:31.168594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da2335-81f4-4f7b-8103-4b52df115d2e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:31.168938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da2335-81f4-4f7b-8103-4b52df115d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:31.169746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17da2335-81f4-4f7b-8103-4b52df115d2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:31.170485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da2335-81f4-4f7b-8103-4b52df115d2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:31.183516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17da2335-81f4-4f7b-8103-4b52df115d2e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:31.291220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:31.301444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 893/1785] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:31 2022] POST /identity/v3/projects => generated 385 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:31.313027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab5074-d168-4e2a-a437-afcc91d2b4a0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:31.468979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57db8e7-47c2-44f5-9619-98dd26e423b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:31.469652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db8e7-47c2-44f5-9619-98dd26e423b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:31.469933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db8e7-47c2-44f5-9619-98dd26e423b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:31.470197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57db8e7-47c2-44f5-9619-98dd26e423b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:31.470854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57db8e7-47c2-44f5-9619-98dd26e423b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:31.476579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57db8e7-47c2-44f5-9619-98dd26e423b9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:31.534193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 893/1786] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:31 2022] POST /identity/v3/users => generated 293 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:31.538783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da2335-81f4-4f7b-8103-4b52df115d2e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:31.550710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:31.565715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17da2335-81f4-4f7b-8103-4b52df115d2e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:31.574809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f259685-054b-4df1-a890-00b9fe6118cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:31.575655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f259685-054b-4df1-a890-00b9fe6118cc admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:31.575941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f259685-054b-4df1-a890-00b9fe6118cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:31.576217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f259685-054b-4df1-a890-00b9fe6118cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:31.576934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f259685-054b-4df1-a890-00b9fe6118cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:31.583373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f259685-054b-4df1-a890-00b9fe6118cc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:31.617188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 894/1787] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:11:31.621500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57db8e7-47c2-44f5-9619-98dd26e423b9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:31.642001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57db8e7-47c2-44f5-9619-98dd26e423b9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:31.669113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd3e887-ce9a-4cfd-89f1-eab1e23eb141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:31.671611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3e887-ce9a-4cfd-89f1-eab1e23eb141 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:31.671611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3e887-ce9a-4cfd-89f1-eab1e23eb141 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:31.671611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd3e887-ce9a-4cfd-89f1-eab1e23eb141 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:31.674063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3e887-ce9a-4cfd-89f1-eab1e23eb141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:31.680058 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd3e887-ce9a-4cfd-89f1-eab1e23eb141 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:31.730178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f259685-054b-4df1-a890-00b9fe6118cc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:31.732632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 894/1788] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:31.745090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f259685-054b-4df1-a890-00b9fe6118cc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:31.751462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:31.752180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:31.753857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:31.754175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] PATH_INFO: `/v3/projects/6ea909baa62d43b78bb0d6e282a641b6/users/9fb02541f77c49beb44aa4e13304f7d7/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:31.771745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ea909baa62d43b78bb0d6e282a641b6, user_id=9fb02541f77c49beb44aa4e13304f7d7, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:31.776911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:31.805683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 895/1789] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:31 2022] PUT /identity/v3/projects/6ea909baa62d43b78bb0d6e282a641b6/users/9fb02541f77c49beb44aa4e13304f7d7/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:31.809358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0bb3-d492-4d2f-b3ee-a73d63bdde3f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:31.809661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0bb3-d492-4d2f-b3ee-a73d63bdde3f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:31.809981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b0bb3-d492-4d2f-b3ee-a73d63bdde3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:31.858555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-343b0bb3-d492-4d2f-b3ee-a73d63bdde3f None None] MFA Rules not processed for user `9fb02541f77c49beb44aa4e13304f7d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:31.859909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-343b0bb3-d492-4d2f-b3ee-a73d63bdde3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:31.970037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 895/1790] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:31.986480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:31.988829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:32.074652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dec0f0-5cd4-48df-b094-682e032cd972 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:32.079064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b0f95-c45b-4e53-b62b-95df47cd5ef2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:32.079773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b0f95-c45b-4e53-b62b-95df47cd5ef2 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:32.080138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b0f95-c45b-4e53-b62b-95df47cd5ef2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:32.080782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b0f95-c45b-4e53-b62b-95df47cd5ef2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:32.100972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b0f95-c45b-4e53-b62b-95df47cd5ef2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:32.102727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b0f95-c45b-4e53-b62b-95df47cd5ef2 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:32.113945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 896/1791] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:32.139618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b0bb3-d492-4d2f-b3ee-a73d63bdde3f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:32.396188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:32.425121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b0bb3-d492-4d2f-b3ee-a73d63bdde3f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:32.433611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:32.435424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:32.436055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:32.436542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] PATH_INFO: `/v3/users/9fb02541f77c49beb44aa4e13304f7d7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:32.438573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb02541f77c49beb44aa4e13304f7d7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:32.443279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:32.583706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb02541f77c49beb44aa4e13304f7d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:32.610241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb02541f77c49beb44aa4e13304f7d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:32.611022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb02541f77c49beb44aa4e13304f7d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:32.621659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb02541f77c49beb44aa4e13304f7d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:32.648036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb02541f77c49beb44aa4e13304f7d7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:32.660360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 896/1792] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:32 2022] DELETE /identity/v3/users/9fb02541f77c49beb44aa4e13304f7d7 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:32.665607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17644b-0929-4d0c-9fc4-9bafb2910cad None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:32.666203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17644b-0929-4d0c-9fc4-9bafb2910cad None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:32.666621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab17644b-0929-4d0c-9fc4-9bafb2910cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:32.702067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ab17644b-0929-4d0c-9fc4-9bafb2910cad None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:32.703688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab17644b-0929-4d0c-9fc4-9bafb2910cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:32.791592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 897/1793] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:32.798602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:32.800088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:32.950595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95f1002-065d-4af7-92ce-603418f083ab admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:32.954565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c5db7c-c074-48a9-b92f-b407a4785f1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:32.955271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5db7c-c074-48a9-b92f-b407a4785f1e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:32.955641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5db7c-c074-48a9-b92f-b407a4785f1e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:32.955988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c5db7c-c074-48a9-b92f-b407a4785f1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:32.971572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5db7c-c074-48a9-b92f-b407a4785f1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:32.973485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c5db7c-c074-48a9-b92f-b407a4785f1e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:32.990837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 897/1794] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:33.667028 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17644b-0929-4d0c-9fc4-9bafb2910cad None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:33.669129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ab17644b-0929-4d0c-9fc4-9bafb2910cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:33.694235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab17644b-0929-4d0c-9fc4-9bafb2910cad None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:33.700345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:33.701177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:33.702265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:33.702696 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] PATH_INFO: `/v3/projects/6ea909baa62d43b78bb0d6e282a641b6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:33.704049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ea909baa62d43b78bb0d6e282a641b6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:33.706369 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:33.760779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ea909baa62d43b78bb0d6e282a641b6', 'request_id': 'req-58485a6b-4696-4f97-9aaf-ff3d094a04f6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:33.766629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ea909baa62d43b78bb0d6e282a641b6', 'request_id': 'req-58485a6b-4696-4f97-9aaf-ff3d094a04f6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:33.773104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ea909baa62d43b78bb0d6e282a641b6', 'request_id': 'req-58485a6b-4696-4f97-9aaf-ff3d094a04f6'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:33.775937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:33.790058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] The token cache is being invalidate because project 6ea909baa62d43b78bb0d6e282a641b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:33.790364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:33.791141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-58485a6b-4696-4f97-9aaf-ff3d094a04f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:33.792722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 898/1795] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:33 2022] DELETE /identity/v3/projects/6ea909baa62d43b78bb0d6e282a641b6 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:33.797841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbce90-4963-4441-81c0-1bd94380fd75 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:33.798800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbce90-4963-4441-81c0-1bd94380fd75 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:33.799075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbce90-4963-4441-81c0-1bd94380fd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:33.825170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7afbce90-4963-4441-81c0-1bd94380fd75 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:33.826486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7afbce90-4963-4441-81c0-1bd94380fd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:33.907895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 898/1796] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:33.915687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7e7a-eb1e-406b-bce0-8bcaa6a00afc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:33.916271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7e7a-eb1e-406b-bce0-8bcaa6a00afc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:33.918656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c7e7a-eb1e-406b-bce0-8bcaa6a00afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:33.948011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-076c7e7a-eb1e-406b-bce0-8bcaa6a00afc None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:33.949502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-076c7e7a-eb1e-406b-bce0-8bcaa6a00afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:33.965800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 899/1797] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:33.967658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbce90-4963-4441-81c0-1bd94380fd75 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:33.991854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afbce90-4963-4441-81c0-1bd94380fd75 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:34.006030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:34.006814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:34.007190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:34.007560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:34.008347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:34.008783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:34.010587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:34.028591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 899/1798] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:34.032718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c7e7a-eb1e-406b-bce0-8bcaa6a00afc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:34.044255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c7e7a-eb1e-406b-bce0-8bcaa6a00afc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:34.050562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb0f9c-a68d-4a6d-b2ae-e71e80445043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:34.055411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb0f9c-a68d-4a6d-b2ae-e71e80445043 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:34.055886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb0f9c-a68d-4a6d-b2ae-e71e80445043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:34.056325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb0f9c-a68d-4a6d-b2ae-e71e80445043 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:34.057778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb0f9c-a68d-4a6d-b2ae-e71e80445043 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:34.060519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb0f9c-a68d-4a6d-b2ae-e71e80445043 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:34.103999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 900/1799] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:34 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:34.108509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:34.119049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e44dc3-8301-4549-8877-33563d998c25 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:34.123348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:34.203778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf841a5-d15f-4a4d-826f-90dcb15fa13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:34.204696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf841a5-d15f-4a4d-826f-90dcb15fa13b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:34.205555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf841a5-d15f-4a4d-826f-90dcb15fa13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:34.205555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf841a5-d15f-4a4d-826f-90dcb15fa13b admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:34.206135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf841a5-d15f-4a4d-826f-90dcb15fa13b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:34.208149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf841a5-d15f-4a4d-826f-90dcb15fa13b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:34.245040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 900/1800] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:34 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:34.250728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb0f9c-a68d-4a6d-b2ae-e71e80445043 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:34.263681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb0f9c-a68d-4a6d-b2ae-e71e80445043 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:34.269759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19075fcf-bb87-4da9-b713-1db2014173a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:34.270462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075fcf-bb87-4da9-b713-1db2014173a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:34.270746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075fcf-bb87-4da9-b713-1db2014173a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:34.271023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19075fcf-bb87-4da9-b713-1db2014173a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:34.271755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19075fcf-bb87-4da9-b713-1db2014173a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:34.273428 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19075fcf-bb87-4da9-b713-1db2014173a5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:34.285472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 901/1801] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:34.289446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf841a5-d15f-4a4d-826f-90dcb15fa13b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:34.302239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf841a5-d15f-4a4d-826f-90dcb15fa13b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:34.307513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f79c3-7e24-4101-92bc-188b899bed45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:34.308344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f79c3-7e24-4101-92bc-188b899bed45 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:34.308693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f79c3-7e24-4101-92bc-188b899bed45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:34.309032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f79c3-7e24-4101-92bc-188b899bed45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:34.309810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f79c3-7e24-4101-92bc-188b899bed45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:34.311639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f79c3-7e24-4101-92bc-188b899bed45 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:34.325963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 901/1802] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:11:34.329957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19075fcf-bb87-4da9-b713-1db2014173a5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:34.350015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19075fcf-bb87-4da9-b713-1db2014173a5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:34.360496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:34.361580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:34.362165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:34.362548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] PATH_INFO: `/v3/projects/652d9098b8e5436fbc0542cd8f8cffbe/users/9a532740bc794f89b02bf612de9d61ff/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:34.379303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=652d9098b8e5436fbc0542cd8f8cffbe, user_id=9a532740bc794f89b02bf612de9d61ff, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:34.382284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:34.398720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 902/1803] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:34 2022] PUT /identity/v3/projects/652d9098b8e5436fbc0542cd8f8cffbe/users/9a532740bc794f89b02bf612de9d61ff/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:34.409423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c28ca-d1fd-46ff-8cf4-12351f432ab7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:34.409803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c28ca-d1fd-46ff-8cf4-12351f432ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:34.410110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c28ca-d1fd-46ff-8cf4-12351f432ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:34.440940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-892c28ca-d1fd-46ff-8cf4-12351f432ab7 None None] MFA Rules not processed for user `9a532740bc794f89b02bf612de9d61ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:34.442270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-892c28ca-d1fd-46ff-8cf4-12351f432ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:34.526713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 902/1804] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:34 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:34.549874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:34.551815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:34.630798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d3d69e-8439-4789-a37f-cba6d39deb63 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:34.635092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f617d-020f-47af-bab6-9b54f968cd85 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417b661f770846f189c36d0804b6dce5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:34.636033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f617d-020f-47af-bab6-9b54f968cd85 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:34.636406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f617d-020f-47af-bab6-9b54f968cd85 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:34.636672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f617d-020f-47af-bab6-9b54f968cd85 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:34.647392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f617d-020f-47af-bab6-9b54f968cd85 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:34.648683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f617d-020f-47af-bab6-9b54f968cd85 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:34.659421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 903/1805] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:37.017845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dcc4c-3ab9-47b1-9592-ebf2e1da9967 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:37.017845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dcc4c-3ab9-47b1-9592-ebf2e1da9967 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:37.018381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6dcc4c-3ab9-47b1-9592-ebf2e1da9967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:37.117855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ff6dcc4c-3ab9-47b1-9592-ebf2e1da9967 None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:37.119164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ff6dcc4c-3ab9-47b1-9592-ebf2e1da9967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:37.172911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8eeec-4746-4044-8cb5-287ba62fbd30 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:37.173864 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8eeec-4746-4044-8cb5-287ba62fbd30 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:37.174399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e8eeec-4746-4044-8cb5-287ba62fbd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:37.222143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c7e8eeec-4746-4044-8cb5-287ba62fbd30 None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:37.224799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:37.225928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7e8eeec-4746-4044-8cb5-287ba62fbd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:37.282447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 903/1806] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:11:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:37.321300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 904/1807] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:11:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:11:37.439388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dcc4c-3ab9-47b1-9592-ebf2e1da9967 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:37.442048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ff6dcc4c-3ab9-47b1-9592-ebf2e1da9967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:37.450154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8eeec-4746-4044-8cb5-287ba62fbd30 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:37.451476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c7e8eeec-4746-4044-8cb5-287ba62fbd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:37.474878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:37.573769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e8eeec-4746-4044-8cb5-287ba62fbd30 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:37.584882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6dcc4c-3ab9-47b1-9592-ebf2e1da9967 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:37.587343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:37.588089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:37.588425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:37.588802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:37.591920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:37.592719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:37.593137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:37.593544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:37.598190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:37.599986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:37.606640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:37.608633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:37.610576 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 905/1808] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:11:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:11:37.624227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 904/1809] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:11:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:11:42.915415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:42.917244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:43.014769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ebcdd1-b5f6-4d47-b96f-35f3756be2f0 service swift] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:43.019904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deffb564-5330-4a45-a5b2-0986f2c1b7e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:43.020552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb564-5330-4a45-a5b2-0986f2c1b7e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:43.020846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb564-5330-4a45-a5b2-0986f2c1b7e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:43.021171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deffb564-5330-4a45-a5b2-0986f2c1b7e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:43.023710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-deffb564-5330-4a45-a5b2-0986f2c1b7e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:43.140878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffb564-5330-4a45-a5b2-0986f2c1b7e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:43.143181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deffb564-5330-4a45-a5b2-0986f2c1b7e4 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:43.155165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 906/1810] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:42 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:44.388419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:44.390040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:44.471566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8cc9be-34e7-4b02-956b-98848ca36e69 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:44.478641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:44.479617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:44.479896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:44.480197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] PATH_INFO: `/v3/users/960f0555013448d4af67e9a1a6e2582a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:44.481363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=960f0555013448d4af67e9a1a6e2582a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:44.485709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:44.543016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '960f0555013448d4af67e9a1a6e2582a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:44.551091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '960f0555013448d4af67e9a1a6e2582a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:44.551913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '960f0555013448d4af67e9a1a6e2582a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:44.555028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:44.561689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '960f0555013448d4af67e9a1a6e2582a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:44.575992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '960f0555013448d4af67e9a1a6e2582a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:44.582226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 905/1811] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:44 2022] DELETE /identity/v3/users/960f0555013448d4af67e9a1a6e2582a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:44.586993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860b31a-4665-4c08-b47c-bafd971cafc7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:44.587475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860b31a-4665-4c08-b47c-bafd971cafc7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:44.587575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3860b31a-4665-4c08-b47c-bafd971cafc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:44.608623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3860b31a-4665-4c08-b47c-bafd971cafc7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:44.609922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3860b31a-4665-4c08-b47c-bafd971cafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:44.645253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:44.691051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 907/1812] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:44.701504 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:44.703011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:44.806453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7b81f-fc42-4f79-8bb9-1d142462ae06 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:44.810239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4fa11-b7da-4299-bb90-4dc039fcfcea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:44.810916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fa11-b7da-4299-bb90-4dc039fcfcea service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:44.811192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fa11-b7da-4299-bb90-4dc039fcfcea service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:44.811457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4fa11-b7da-4299-bb90-4dc039fcfcea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:44.831173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4fa11-b7da-4299-bb90-4dc039fcfcea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:44.832890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4fa11-b7da-4299-bb90-4dc039fcfcea service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:44.845563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 906/1813] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:11:45.137819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860b31a-4665-4c08-b47c-bafd971cafc7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:45.139320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3860b31a-4665-4c08-b47c-bafd971cafc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:45.152200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3860b31a-4665-4c08-b47c-bafd971cafc7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:45.169437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:45.170160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:45.170433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:45.170708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] PATH_INFO: `/v3/projects/07b623ce970f4ac3891ec87390a0513f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:45.172093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] RBAC: Authorizing `identity:delete_project(project_id=07b623ce970f4ac3891ec87390a0513f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:45.174358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:45.225092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07b623ce970f4ac3891ec87390a0513f', 'request_id': 'req-16272685-5845-463d-9626-c6327a4aa46d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.231276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07b623ce970f4ac3891ec87390a0513f', 'request_id': 'req-16272685-5845-463d-9626-c6327a4aa46d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.237725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07b623ce970f4ac3891ec87390a0513f', 'request_id': 'req-16272685-5845-463d-9626-c6327a4aa46d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.263640 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] The token cache is being invalidate because project 07b623ce970f4ac3891ec87390a0513f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:45.263936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.264592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.266623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 908/1814] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:45 2022] DELETE /identity/v3/projects/07b623ce970f4ac3891ec87390a0513f => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:45.270547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4fa11-b7da-4299-bb90-4dc039fcfcea service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:45.281497 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4fa11-b7da-4299-bb90-4dc039fcfcea service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:45.357989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:45.358775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:45.359119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:45.359459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] PATH_INFO: `/v3/users/aa3ba9d72366488396952e1ec8d94355` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:45.360836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3ba9d72366488396952e1ec8d94355)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:45.363026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:45.420253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3ba9d72366488396952e1ec8d94355'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.427932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3ba9d72366488396952e1ec8d94355'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.430095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3ba9d72366488396952e1ec8d94355'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.449193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3ba9d72366488396952e1ec8d94355'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.464624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b2288eb2-af2b-494b-a00b-32db462d5493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3ba9d72366488396952e1ec8d94355'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:45.466104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:45.472064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 907/1815] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:11:45 2022] DELETE /identity/v3/users/aa3ba9d72366488396952e1ec8d94355 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:45.856226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:45.857986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:45.898504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16272685-5845-463d-9626-c6327a4aa46d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.001054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.001732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.002021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.002285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] PATH_INFO: `/v3/projects/20e11a21c67640d898145f40b6df4d0e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.003402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e11a21c67640d898145f40b6df4d0e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.005314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.059895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e11a21c67640d898145f40b6df4d0e', 'request_id': 'req-09088f3b-0e1f-4ee8-883a-10d9e14269dc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:46.065208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e11a21c67640d898145f40b6df4d0e', 'request_id': 'req-09088f3b-0e1f-4ee8-883a-10d9e14269dc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:46.071338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e11a21c67640d898145f40b6df4d0e', 'request_id': 'req-09088f3b-0e1f-4ee8-883a-10d9e14269dc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:46.073003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:46.086348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] The token cache is being invalidate because project 20e11a21c67640d898145f40b6df4d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:11:46.086697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:46.087690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-09088f3b-0e1f-4ee8-883a-10d9e14269dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:11:46.089265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 909/1816] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:11:45 2022] DELETE /identity/v3/projects/20e11a21c67640d898145f40b6df4d0e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:46.095374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e0e0a-a2e5-4655-9b89-0ab48290c7d7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.095662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e0e0a-a2e5-4655-9b89-0ab48290c7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.095929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e0e0a-a2e5-4655-9b89-0ab48290c7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.118128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-934e0e0a-a2e5-4655-9b89-0ab48290c7d7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:46.119249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-934e0e0a-a2e5-4655-9b89-0ab48290c7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:46.179102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 908/1817] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:46.187447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b76f59-af1d-4025-8d70-5a2ed4afcd7a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.187723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b76f59-af1d-4025-8d70-5a2ed4afcd7a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.187989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b76f59-af1d-4025-8d70-5a2ed4afcd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.208220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-19b76f59-af1d-4025-8d70-5a2ed4afcd7a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:46.209424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-19b76f59-af1d-4025-8d70-5a2ed4afcd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:46.218449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 910/1818] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:46.224337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e0e0a-a2e5-4655-9b89-0ab48290c7d7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.243119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e0e0a-a2e5-4655-9b89-0ab48290c7d7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.249968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.250820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.251100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.251618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.252713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:11:46.253117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.255123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.269252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 909/1819] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:11:46.274001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b76f59-af1d-4025-8d70-5a2ed4afcd7a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.284034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b76f59-af1d-4025-8d70-5a2ed4afcd7a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.290014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f08cce-8b4e-4ff8-8382-124ea3a5eec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.290656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f08cce-8b4e-4ff8-8382-124ea3a5eec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.291226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f08cce-8b4e-4ff8-8382-124ea3a5eec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.291495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f08cce-8b4e-4ff8-8382-124ea3a5eec4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.292150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f08cce-8b4e-4ff8-8382-124ea3a5eec4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.294210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f08cce-8b4e-4ff8-8382-124ea3a5eec4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.323863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 911/1820] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:46 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:46.328381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.339310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8f4dda-cc6c-424f-91e4-c93a0efd2811 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.398627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb91837-6356-49fc-a88e-314479da094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.399253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb91837-6356-49fc-a88e-314479da094c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.399527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb91837-6356-49fc-a88e-314479da094c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.399793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb91837-6356-49fc-a88e-314479da094c admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.400433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb91837-6356-49fc-a88e-314479da094c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.402440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb91837-6356-49fc-a88e-314479da094c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.425928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 910/1821] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:46 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:46.430202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f08cce-8b4e-4ff8-8382-124ea3a5eec4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.441458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f08cce-8b4e-4ff8-8382-124ea3a5eec4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.447224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370713-9d99-433b-acc1-76a0ca179bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.447833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370713-9d99-433b-acc1-76a0ca179bce admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.448108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370713-9d99-433b-acc1-76a0ca179bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.448373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f370713-9d99-433b-acc1-76a0ca179bce admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.449117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f370713-9d99-433b-acc1-76a0ca179bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.450764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f370713-9d99-433b-acc1-76a0ca179bce admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.460983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 912/1822] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:46.465147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb91837-6356-49fc-a88e-314479da094c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.475113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb91837-6356-49fc-a88e-314479da094c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.480199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be232b9c-7887-4c9c-b4b7-2d6eeb69c12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.480814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232b9c-7887-4c9c-b4b7-2d6eeb69c12c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.481085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232b9c-7887-4c9c-b4b7-2d6eeb69c12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.481330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be232b9c-7887-4c9c-b4b7-2d6eeb69c12c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.482031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be232b9c-7887-4c9c-b4b7-2d6eeb69c12c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.483614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be232b9c-7887-4c9c-b4b7-2d6eeb69c12c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.494091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 911/1823] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:46.498023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370713-9d99-433b-acc1-76a0ca179bce admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.510931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f370713-9d99-433b-acc1-76a0ca179bce admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.517041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbf482-e5a9-44e3-a207-47808982550f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.517754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbf482-e5a9-44e3-a207-47808982550f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.518344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbf482-e5a9-44e3-a207-47808982550f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.518344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cbf482-e5a9-44e3-a207-47808982550f admin admin] PATH_INFO: `/v3/projects/b5bd1ac9808a4eb79f6b48bdf93c24c9/users/1fb013e26c664e8f8ef302accd8baee5/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.534303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbf482-e5a9-44e3-a207-47808982550f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5bd1ac9808a4eb79f6b48bdf93c24c9, user_id=1fb013e26c664e8f8ef302accd8baee5, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.537037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cbf482-e5a9-44e3-a207-47808982550f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.551815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 913/1824] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:46 2022] PUT /identity/v3/projects/b5bd1ac9808a4eb79f6b48bdf93c24c9/users/1fb013e26c664e8f8ef302accd8baee5/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:46.562540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755499-5a6b-4aba-ab16-5ac790113810 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.562814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755499-5a6b-4aba-ab16-5ac790113810 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.563074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755499-5a6b-4aba-ab16-5ac790113810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.599522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-df755499-5a6b-4aba-ab16-5ac790113810 None None] MFA Rules not processed for user `1fb013e26c664e8f8ef302accd8baee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:46.601109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-df755499-5a6b-4aba-ab16-5ac790113810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:46.662342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:46.675099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 912/1825] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:11:46.680103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbf482-e5a9-44e3-a207-47808982550f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.741621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cbf482-e5a9-44e3-a207-47808982550f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.746615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af29cb9-53b1-43a5-ae9f-235df0d084af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.747233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af29cb9-53b1-43a5-ae9f-235df0d084af admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.747568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af29cb9-53b1-43a5-ae9f-235df0d084af admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.747846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af29cb9-53b1-43a5-ae9f-235df0d084af admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.748483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af29cb9-53b1-43a5-ae9f-235df0d084af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.750446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af29cb9-53b1-43a5-ae9f-235df0d084af admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.779558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 914/1826] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:11:46 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:46.783663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755499-5a6b-4aba-ab16-5ac790113810 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.794536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755499-5a6b-4aba-ab16-5ac790113810 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.854056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38124c-2a89-41d2-9418-1c3895acf5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.855025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38124c-2a89-41d2-9418-1c3895acf5bb admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.855502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38124c-2a89-41d2-9418-1c3895acf5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.855770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f38124c-2a89-41d2-9418-1c3895acf5bb admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.856418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38124c-2a89-41d2-9418-1c3895acf5bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.858637 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f38124c-2a89-41d2-9418-1c3895acf5bb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.881222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 913/1827] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:11:46 2022] POST /identity/v3/users => generated 298 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:11:46.885522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af29cb9-53b1-43a5-ae9f-235df0d084af admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.896796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af29cb9-53b1-43a5-ae9f-235df0d084af admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.901697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400cbd8-659f-4165-806b-2cc9afaaf82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.903026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400cbd8-659f-4165-806b-2cc9afaaf82e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.903314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400cbd8-659f-4165-806b-2cc9afaaf82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.903587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c400cbd8-659f-4165-806b-2cc9afaaf82e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.904250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400cbd8-659f-4165-806b-2cc9afaaf82e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.906014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c400cbd8-659f-4165-806b-2cc9afaaf82e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.922217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 915/1828] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:46.926241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38124c-2a89-41d2-9418-1c3895acf5bb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:46.937862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f38124c-2a89-41d2-9418-1c3895acf5bb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:46.945478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51223f3f-1c61-4bb4-aa78-674489a07696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:46.946204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223f3f-1c61-4bb4-aa78-674489a07696 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:46.946650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223f3f-1c61-4bb4-aa78-674489a07696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:46.946887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51223f3f-1c61-4bb4-aa78-674489a07696 admin admin] PATH_INFO: `/v3/projects/60f1acdb72b14a9bb7762ab6254f04ec/users/d198e465168145a58103dffcd3d5e555/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:46.978348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51223f3f-1c61-4bb4-aa78-674489a07696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f1acdb72b14a9bb7762ab6254f04ec, user_id=d198e465168145a58103dffcd3d5e555, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:46.988602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51223f3f-1c61-4bb4-aa78-674489a07696 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:46.999173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 914/1829] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:46 2022] PUT /identity/v3/projects/60f1acdb72b14a9bb7762ab6254f04ec/users/d198e465168145a58103dffcd3d5e555/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:47.003301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400cbd8-659f-4165-806b-2cc9afaaf82e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:47.013860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c400cbd8-659f-4165-806b-2cc9afaaf82e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:47.083700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236ded80-1292-4490-8fb1-7efda9e28c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:47.084319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ded80-1292-4490-8fb1-7efda9e28c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:47.084595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ded80-1292-4490-8fb1-7efda9e28c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:47.084861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236ded80-1292-4490-8fb1-7efda9e28c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:47.085525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ded80-1292-4490-8fb1-7efda9e28c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:47.087225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236ded80-1292-4490-8fb1-7efda9e28c06 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:47.089358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:47.098237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 916/1830] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:11:47.102859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51223f3f-1c61-4bb4-aa78-674489a07696 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:47.121402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51223f3f-1c61-4bb4-aa78-674489a07696 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:47.126979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:47.127672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:47.127947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:47.128211 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] PATH_INFO: `/v3/projects/60f1acdb72b14a9bb7762ab6254f04ec/users/d198e465168145a58103dffcd3d5e555/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:47.130315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60f1acdb72b14a9bb7762ab6254f04ec, user_id=d198e465168145a58103dffcd3d5e555, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:47.132968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:47.144311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 915/1831] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:11:47 2022] PUT /identity/v3/projects/60f1acdb72b14a9bb7762ab6254f04ec/users/d198e465168145a58103dffcd3d5e555/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:11:47.155944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e01cf-c69f-4c5e-8edf-58a692b90c38 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:47.156256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e01cf-c69f-4c5e-8edf-58a692b90c38 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:47.156532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e01cf-c69f-4c5e-8edf-58a692b90c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:47.185425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0c4e01cf-c69f-4c5e-8edf-58a692b90c38 None None] MFA Rules not processed for user `d198e465168145a58103dffcd3d5e555`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:11:47.186796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0c4e01cf-c69f-4c5e-8edf-58a692b90c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:47.265145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 917/1832] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:11:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:11:47.273656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:47.275581 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:47.333864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052b879-248a-4197-b2ea-b6bbe1223e84 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:47.337493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e4e93d-be76-45d3-a35f-a5a6a056a0bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:47.338141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e93d-be76-45d3-a35f-a5a6a056a0bf service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:47.338418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e93d-be76-45d3-a35f-a5a6a056a0bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:47.338683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4e93d-be76-45d3-a35f-a5a6a056a0bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:47.348369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4e93d-be76-45d3-a35f-a5a6a056a0bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:47.349861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e4e93d-be76-45d3-a35f-a5a6a056a0bf service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:47.360995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 916/1833] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:47.475174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e01cf-c69f-4c5e-8edf-58a692b90c38 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:11:47.476712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0c4e01cf-c69f-4c5e-8edf-58a692b90c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:11:47.489834 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e01cf-c69f-4c5e-8edf-58a692b90c38 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:11:47.494072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:11:47.494702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:11:47.494982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:11:47.495253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:11:47.558460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:11:47.560116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:11:47.569677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 918/1834] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:11:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:11:57.356166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:11:57.565094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=63 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:38.614302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f57de-5fcd-409b-a491-549eb72a63c1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:38.614965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f57de-5fcd-409b-a491-549eb72a63c1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:38.614965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113f57de-5fcd-409b-a491-549eb72a63c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:38.636436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-113f57de-5fcd-409b-a491-549eb72a63c1 None None] MFA Rules not processed for user `6c9b940cd9e0432daa235f82becb4d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:12:38.637669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-113f57de-5fcd-409b-a491-549eb72a63c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:38.695538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 917/1835] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Oct 5 14:12:38 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:12:38.718082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:12:38.719772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:38.721382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f57de-5fcd-409b-a491-549eb72a63c1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:12:38.723531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-113f57de-5fcd-409b-a491-549eb72a63c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:38.735438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3e84c-3822-489e-bce7-bb52b0573453 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:12:38.735676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113f57de-5fcd-409b-a491-549eb72a63c1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:12:38.739819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fdcc3-ca18-45ad-9388-7900e3b65739 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:12:38.740450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdcc3-ca18-45ad-9388-7900e3b65739 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:38.740727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdcc3-ca18-45ad-9388-7900e3b65739 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:38.740995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fdcc3-ca18-45ad-9388-7900e3b65739 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:38.750027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fdcc3-ca18-45ad-9388-7900e3b65739 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:12:38.751455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627fdcc3-ca18-45ad-9388-7900e3b65739 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:12:38.755691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4232fdc1-80f2-410a-b4e4-e66077abdf1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:12:38.756853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232fdc1-80f2-410a-b4e4-e66077abdf1f service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:38.758039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232fdc1-80f2-410a-b4e4-e66077abdf1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:38.758478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4232fdc1-80f2-410a-b4e4-e66077abdf1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:38.768181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232fdc1-80f2-410a-b4e4-e66077abdf1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:12:38.771562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4232fdc1-80f2-410a-b4e4-e66077abdf1f service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:12:38.777204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 918/1836] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:12:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:12:38.781547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 919/1837] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:12:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:12:39.798079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9debb-fa3f-4e87-a99b-e1fdef4eb641 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:39.798632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9debb-fa3f-4e87-a99b-e1fdef4eb641 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:39.798920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9debb-fa3f-4e87-a99b-e1fdef4eb641 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:39.802602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 919/1838] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 14:12:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:12:39.833948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87828d91-5312-42df-b19a-9a55aacf4550 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:39.834399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87828d91-5312-42df-b19a-9a55aacf4550 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:39.834472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87828d91-5312-42df-b19a-9a55aacf4550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:39.863700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-87828d91-5312-42df-b19a-9a55aacf4550 None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:12:39.868315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-87828d91-5312-42df-b19a-9a55aacf4550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:39.961762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 920/1839] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 14:12:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:12:39.977387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9debb-fa3f-4e87-a99b-e1fdef4eb641 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:12:39.981195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-99c9debb-fa3f-4e87-a99b-e1fdef4eb641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:40.020750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c9debb-fa3f-4e87-a99b-e1fdef4eb641 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:12:40.028873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b87d7a-7c03-46dc-b5da-a6d624f3de9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:12:40.029676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b87d7a-7c03-46dc-b5da-a6d624f3de9a service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:40.030019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b87d7a-7c03-46dc-b5da-a6d624f3de9a service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:40.030354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b87d7a-7c03-46dc-b5da-a6d624f3de9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:40.041316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b87d7a-7c03-46dc-b5da-a6d624f3de9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:12:40.052795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b87d7a-7c03-46dc-b5da-a6d624f3de9a service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:12:40.073047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 920/1840] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:12:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:12:40.291179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6fe5-9098-480a-b0c6-e1ea4a98c6b0 None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:40.291615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6fe5-9098-480a-b0c6-e1ea4a98c6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:40.291980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec6fe5-9098-480a-b0c6-e1ea4a98c6b0 None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:40.293618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 921/1841] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 14:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:12:40.315662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800b81-54a7-4feb-838d-81d96bba613a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:40.315854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800b81-54a7-4feb-838d-81d96bba613a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:40.316219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de800b81-54a7-4feb-838d-81d96bba613a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:40.338727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-de800b81-54a7-4feb-838d-81d96bba613a None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:12:40.340213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-de800b81-54a7-4feb-838d-81d96bba613a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:40.355121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 921/1842] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 14:12:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:12:40.391460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6fe5-9098-480a-b0c6-e1ea4a98c6b0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:12:40.412624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec6fe5-9098-480a-b0c6-e1ea4a98c6b0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:12:40.428261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:12:40.433057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:40.433597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:40.434328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:40.461837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:12:40.471410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:12:40.498152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 922/1843] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:12:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:12:40.641930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8194db-63d0-441a-a92e-d7091042b65a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:40.643117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8194db-63d0-441a-a92e-d7091042b65a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:40.643607 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8194db-63d0-441a-a92e-d7091042b65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:40.673072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4c8194db-63d0-441a-a92e-d7091042b65a None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:12:40.674922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4c8194db-63d0-441a-a92e-d7091042b65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:40.783625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 922/1844] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:12:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:12:40.828611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:12:40.830071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:40.932322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad2ca5-7411-4fd4-8c6c-8b883cb249f2 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:12:40.938368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed186362-57f5-413e-813e-0c898bbe4de7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:12:40.939023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed186362-57f5-413e-813e-0c898bbe4de7 service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:40.939302 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed186362-57f5-413e-813e-0c898bbe4de7 service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:40.939605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed186362-57f5-413e-813e-0c898bbe4de7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:40.949380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed186362-57f5-413e-813e-0c898bbe4de7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:12:40.951142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed186362-57f5-413e-813e-0c898bbe4de7 service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:12:40.961176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 923/1845] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:12:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:12:48.454672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c221e-646d-4351-b076-43c8db443f23 None None] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:48.455357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c221e-646d-4351-b076-43c8db443f23 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:48.455357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c221e-646d-4351-b076-43c8db443f23 None None] PATH_INFO: `/` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:48.457016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 923/1846] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 14:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:12:48.471812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bbfc-a28b-430c-95d1-8413ef1dcaca None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:48.472398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bbfc-a28b-430c-95d1-8413ef1dcaca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:48.472933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bbfc-a28b-430c-95d1-8413ef1dcaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:48.499035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e9f5bbfc-a28b-430c-95d1-8413ef1dcaca None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:12:48.500426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e9f5bbfc-a28b-430c-95d1-8413ef1dcaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:12:48.516867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 924/1847] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 14:12:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:12:48.530896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c221e-646d-4351-b076-43c8db443f23 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:12:48.553163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c221e-646d-4351-b076-43c8db443f23 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:12:48.559206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdffe9-db3a-4f1a-a444-ba452acfa3e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:12:48.559848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdffe9-db3a-4f1a-a444-ba452acfa3e6 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:12:48.560123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdffe9-db3a-4f1a-a444-ba452acfa3e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:12:48.560391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabdffe9-db3a-4f1a-a444-ba452acfa3e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:12:48.569401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabdffe9-db3a-4f1a-a444-ba452acfa3e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:12:48.571026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabdffe9-db3a-4f1a-a444-ba452acfa3e6 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:12:48.580966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 924/1848] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:12:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:12:58.492260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=78 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:12:58.576837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=69 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:15:17.386040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5bbfc-a28b-430c-95d1-8413ef1dcaca None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:15:17.387585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e9f5bbfc-a28b-430c-95d1-8413ef1dcaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:15:17.457973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5bbfc-a28b-430c-95d1-8413ef1dcaca None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:15:17.463905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:15:17.464543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:15:17.464817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:15:17.465079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:15:17.466965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:15:17.478513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:15:17.480102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:15:17.489875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 925/1849] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:15:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:15:27.485731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=32 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:16:20.375668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdffe9-db3a-4f1a-a444-ba452acfa3e6 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:16:20.387336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabdffe9-db3a-4f1a-a444-ba452acfa3e6 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:16:20.391838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:16:20.392449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:16:20.392722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:16:20.392976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:16:20.394997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:16:20.466139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:16:20.467577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:16:20.477156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 925/1850] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:16:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:16:21.875041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:16:21.887846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b147b1-9e3d-4ddb-a998-2b4db5d33c58 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:16:21.892705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:16:21.893315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:16:21.893589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:16:21.893858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:16:21.895966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:16:21.964130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:16:21.965589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:16:21.975245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 926/1851] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:16:21 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:16:25.212885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:16:25.221925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cc59fc-3325-4d22-8a25-03e8d97c8545 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:16:25.226518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:16:25.227136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:16:25.227398 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:16:25.227662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:16:25.229931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:16:25.296985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:16:25.299189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:16:25.312512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 926/1852] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:16:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:16:26.407254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:16:26.416127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae606155-1158-4dd4-a09f-6a89632b8305 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:16:26.420219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:16:26.420840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:16:26.421215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:16:26.421488 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:16:26.423437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:16:26.484173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:16:26.485667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:16:26.494714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 927/1853] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:16:26.698265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:16:26.709521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28818d2e-f5c2-4391-b898-ed3f75450fb8 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:16:26.714276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:16:26.714903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:16:26.715178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:16:26.715449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:16:26.717591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:16:26.780201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:16:26.781683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:16:26.790990 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 927/1854] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:16:29.157477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:16:29.166720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82293698-e2b8-43a6-b8cc-249e030d29e1 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:16:29.170975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:16:29.171517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:16:29.171793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:16:29.172055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:16:29.173996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:16:29.235119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:16:29.236565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:16:29.245718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 928/1855] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:16:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:16:34.582120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:16:34.591436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5288c3-854d-4605-84ff-9e8a19b5887c service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:16:34.595666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:16:34.596300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:16:34.596571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:16:34.596845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:16:34.599114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:16:34.632096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:16:34.658691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:16:34.660119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:16:34.669311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 928/1856] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:16:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:16:39.241568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:16:44.665295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:16:48.090680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:16:48.099873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47e9929-e057-4717-87f7-40ea4d0c8a96 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:16:48.104062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77baa7a1-20ad-4200-bca2-ad106354ce01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:16:48.104749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77baa7a1-20ad-4200-bca2-ad106354ce01 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:16:48.105027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77baa7a1-20ad-4200-bca2-ad106354ce01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:16:48.105298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77baa7a1-20ad-4200-bca2-ad106354ce01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:16:48.117022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77baa7a1-20ad-4200-bca2-ad106354ce01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:16:48.118512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77baa7a1-20ad-4200-bca2-ad106354ce01 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:16:48.127281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 929/1857] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:16:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:16:58.123462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:35.229893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:35.231676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:35.300348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f3f8d-4f25-444d-a154-b490638fb05b service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:35.305535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:35.306127 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:35.306403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:35.306675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] PATH_INFO: `/v3/users/9a532740bc794f89b02bf612de9d61ff` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:35.307838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a532740bc794f89b02bf612de9d61ff)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:35.309795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:35.359892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a532740bc794f89b02bf612de9d61ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.366572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a532740bc794f89b02bf612de9d61ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.367226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a532740bc794f89b02bf612de9d61ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.376640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a532740bc794f89b02bf612de9d61ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.388765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a532740bc794f89b02bf612de9d61ff'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.395445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 929/1858] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:17:35 2022] DELETE /identity/v3/users/9a532740bc794f89b02bf612de9d61ff => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:35.400322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea85c6b-8516-46a7-886f-551283cf38ab None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:35.400842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea85c6b-8516-46a7-886f-551283cf38ab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:35.400842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea85c6b-8516-46a7-886f-551283cf38ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:35.421178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bea85c6b-8516-46a7-886f-551283cf38ab None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:35.422430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bea85c6b-8516-46a7-886f-551283cf38ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:35.489513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 930/1859] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:35.499852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:35.501582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:35.563944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e8bc62-2bd2-4807-9021-463476b928f2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:35.567917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8087ada-7082-4c4f-9f8d-0c8dc9c1645f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:35.568523 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087ada-7082-4c4f-9f8d-0c8dc9c1645f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:35.568800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087ada-7082-4c4f-9f8d-0c8dc9c1645f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:35.569065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087ada-7082-4c4f-9f8d-0c8dc9c1645f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:35.579233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8087ada-7082-4c4f-9f8d-0c8dc9c1645f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:35.581477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8087ada-7082-4c4f-9f8d-0c8dc9c1645f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:35.595351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 930/1860] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:35.879782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea85c6b-8516-46a7-886f-551283cf38ab None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:35.881682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bea85c6b-8516-46a7-886f-551283cf38ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:35.904587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea85c6b-8516-46a7-886f-551283cf38ab None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:35.911104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:35.911746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:35.912018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:35.912282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] PATH_INFO: `/v3/projects/652d9098b8e5436fbc0542cd8f8cffbe` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:35.913490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=652d9098b8e5436fbc0542cd8f8cffbe)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:35.915520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:35.963912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '652d9098b8e5436fbc0542cd8f8cffbe', 'request_id': 'req-761c3aed-d9b2-4f5f-8c06-3e06329bb569'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.969317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '652d9098b8e5436fbc0542cd8f8cffbe', 'request_id': 'req-761c3aed-d9b2-4f5f-8c06-3e06329bb569'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.976414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '652d9098b8e5436fbc0542cd8f8cffbe', 'request_id': 'req-761c3aed-d9b2-4f5f-8c06-3e06329bb569'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.991448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] The token cache is being invalidate because project 652d9098b8e5436fbc0542cd8f8cffbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:17:35.991730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.992357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-761c3aed-d9b2-4f5f-8c06-3e06329bb569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:35.993906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 931/1861] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:17:35 2022] DELETE /identity/v3/projects/652d9098b8e5436fbc0542cd8f8cffbe => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:35.999855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef67ea-e12c-41d2-a4fd-a94249ba66a9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.000136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef67ea-e12c-41d2-a4fd-a94249ba66a9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.000405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef67ea-e12c-41d2-a4fd-a94249ba66a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.020628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-daef67ea-e12c-41d2-a4fd-a94249ba66a9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:36.021872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-daef67ea-e12c-41d2-a4fd-a94249ba66a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:36.039862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:36.084474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 931/1862] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:17:36.093466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90bd70-267a-4d04-9912-cf9171947999 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.093832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90bd70-267a-4d04-9912-cf9171947999 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.094216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90bd70-267a-4d04-9912-cf9171947999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.122485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4d90bd70-267a-4d04-9912-cf9171947999 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:36.123719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d90bd70-267a-4d04-9912-cf9171947999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:36.133626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 932/1863] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:36.137652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef67ea-e12c-41d2-a4fd-a94249ba66a9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.147347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef67ea-e12c-41d2-a4fd-a94249ba66a9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.151900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.152499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.152769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.153035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.153699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:17:36.153997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.155561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.169254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 932/1864] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:17:36.171600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90bd70-267a-4d04-9912-cf9171947999 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.181471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90bd70-267a-4d04-9912-cf9171947999 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.189091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c66dee-7a30-4596-b9e0-64a1eecd77c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.189728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66dee-7a30-4596-b9e0-64a1eecd77c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.189992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66dee-7a30-4596-b9e0-64a1eecd77c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.190333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c66dee-7a30-4596-b9e0-64a1eecd77c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.190989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c66dee-7a30-4596-b9e0-64a1eecd77c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.205672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c66dee-7a30-4596-b9e0-64a1eecd77c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.234527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 933/1865] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:36 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:36.238532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.248462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66b7271-164b-4c16-9133-d7889195acb4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.303319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42763292-39bf-435a-a56b-a7287a55fe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.303915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763292-39bf-435a-a56b-a7287a55fe39 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.304187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763292-39bf-435a-a56b-a7287a55fe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.304455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42763292-39bf-435a-a56b-a7287a55fe39 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.305090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42763292-39bf-435a-a56b-a7287a55fe39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.307073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42763292-39bf-435a-a56b-a7287a55fe39 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.330761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 933/1866] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:36 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:36.334920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c66dee-7a30-4596-b9e0-64a1eecd77c6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.345300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c66dee-7a30-4596-b9e0-64a1eecd77c6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.351175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f574-4f96-4269-83f7-c5728cc53d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.351771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f574-4f96-4269-83f7-c5728cc53d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.352041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f574-4f96-4269-83f7-c5728cc53d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.352359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9f574-4f96-4269-83f7-c5728cc53d76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.353045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f574-4f96-4269-83f7-c5728cc53d76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.354665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de9f574-4f96-4269-83f7-c5728cc53d76 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.364626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 934/1867] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:36.368731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42763292-39bf-435a-a56b-a7287a55fe39 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.379000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42763292-39bf-435a-a56b-a7287a55fe39 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.384868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b260e80-f57c-4629-b224-f96e5b3318b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.385464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b260e80-f57c-4629-b224-f96e5b3318b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.385773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b260e80-f57c-4629-b224-f96e5b3318b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.386043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b260e80-f57c-4629-b224-f96e5b3318b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.386690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b260e80-f57c-4629-b224-f96e5b3318b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.388279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b260e80-f57c-4629-b224-f96e5b3318b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.399251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 934/1868] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:36.403961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f574-4f96-4269-83f7-c5728cc53d76 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.414175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de9f574-4f96-4269-83f7-c5728cc53d76 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.419354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf1d3-933e-49dc-b1d6-2666cc2f01b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.419996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf1d3-933e-49dc-b1d6-2666cc2f01b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.420266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf1d3-933e-49dc-b1d6-2666cc2f01b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.420537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ebf1d3-933e-49dc-b1d6-2666cc2f01b0 admin admin] PATH_INFO: `/v3/projects/d46197f77cf946b690cc5c4bf69d45e9/users/083c32f884054b7897702ac2735df8cd/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.436035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebf1d3-933e-49dc-b1d6-2666cc2f01b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d46197f77cf946b690cc5c4bf69d45e9, user_id=083c32f884054b7897702ac2735df8cd, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.438784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ebf1d3-933e-49dc-b1d6-2666cc2f01b0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.453192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 935/1869] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:36 2022] PUT /identity/v3/projects/d46197f77cf946b690cc5c4bf69d45e9/users/083c32f884054b7897702ac2735df8cd/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:17:36.464965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca026c-2d45-4a50-bb1b-3569207dee13 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.465247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca026c-2d45-4a50-bb1b-3569207dee13 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.465514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca026c-2d45-4a50-bb1b-3569207dee13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.494357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-5fca026c-2d45-4a50-bb1b-3569207dee13 None None] MFA Rules not processed for user `083c32f884054b7897702ac2735df8cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:36.495570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5fca026c-2d45-4a50-bb1b-3569207dee13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:36.562520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 935/1870] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:36 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:36.567147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf1d3-933e-49dc-b1d6-2666cc2f01b0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.591471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:36.635698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ebf1d3-933e-49dc-b1d6-2666cc2f01b0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.641211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce89aa-8662-4264-94bb-b6da014a8776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.641857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce89aa-8662-4264-94bb-b6da014a8776 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.642127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce89aa-8662-4264-94bb-b6da014a8776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.642401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce89aa-8662-4264-94bb-b6da014a8776 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.643022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce89aa-8662-4264-94bb-b6da014a8776 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.645039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce89aa-8662-4264-94bb-b6da014a8776 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.673433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 936/1871] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:36 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:36.677580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca026c-2d45-4a50-bb1b-3569207dee13 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.689168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fca026c-2d45-4a50-bb1b-3569207dee13 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.701997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:36.749908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628699d6-7a5d-405d-8013-23f0a59fb40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.750520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628699d6-7a5d-405d-8013-23f0a59fb40c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.750791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628699d6-7a5d-405d-8013-23f0a59fb40c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.751989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628699d6-7a5d-405d-8013-23f0a59fb40c admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.752623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628699d6-7a5d-405d-8013-23f0a59fb40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.754860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628699d6-7a5d-405d-8013-23f0a59fb40c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.778485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 936/1872] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:36 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:36.783588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce89aa-8662-4264-94bb-b6da014a8776 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.794883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce89aa-8662-4264-94bb-b6da014a8776 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.799736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31aa41-de89-45c3-bd35-58d55162439b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.800355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31aa41-de89-45c3-bd35-58d55162439b admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.800679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31aa41-de89-45c3-bd35-58d55162439b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.800889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd31aa41-de89-45c3-bd35-58d55162439b admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.801535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31aa41-de89-45c3-bd35-58d55162439b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.803176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd31aa41-de89-45c3-bd35-58d55162439b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.814129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 937/1873] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:36.818160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628699d6-7a5d-405d-8013-23f0a59fb40c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.828513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628699d6-7a5d-405d-8013-23f0a59fb40c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.833298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37906fd-c6fe-48ac-9bac-306c78e8202f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.833938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37906fd-c6fe-48ac-9bac-306c78e8202f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.834212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37906fd-c6fe-48ac-9bac-306c78e8202f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.834476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37906fd-c6fe-48ac-9bac-306c78e8202f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.835120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37906fd-c6fe-48ac-9bac-306c78e8202f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.836782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37906fd-c6fe-48ac-9bac-306c78e8202f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.846767 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 937/1874] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:36.851047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31aa41-de89-45c3-bd35-58d55162439b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:36.860890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd31aa41-de89-45c3-bd35-58d55162439b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:36.865935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029a262-fbf9-4987-b4ce-41e2d429a03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:36.866604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029a262-fbf9-4987-b4ce-41e2d429a03c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.866866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029a262-fbf9-4987-b4ce-41e2d429a03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.867135 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3029a262-fbf9-4987-b4ce-41e2d429a03c admin admin] PATH_INFO: `/v3/projects/e7156d160e8944209bf025d5adca1ef2/users/1e76b3b679b94888a7f4f7bb956be821/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.882067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029a262-fbf9-4987-b4ce-41e2d429a03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7156d160e8944209bf025d5adca1ef2, user_id=1e76b3b679b94888a7f4f7bb956be821, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:36.884781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3029a262-fbf9-4987-b4ce-41e2d429a03c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:36.899137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 938/1875] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:36 2022] PUT /identity/v3/projects/e7156d160e8944209bf025d5adca1ef2/users/1e76b3b679b94888a7f4f7bb956be821/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:36.909315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb773a0-1956-4bd8-a628-021e03693c75 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:36.909724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb773a0-1956-4bd8-a628-021e03693c75 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:36.910081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb773a0-1956-4bd8-a628-021e03693c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:36.939499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9eb773a0-1956-4bd8-a628-021e03693c75 None None] MFA Rules not processed for user `1e76b3b679b94888a7f4f7bb956be821`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:36.940737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9eb773a0-1956-4bd8-a628-021e03693c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:37.007303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 938/1876] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:37.011948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029a262-fbf9-4987-b4ce-41e2d429a03c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:37.073838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3029a262-fbf9-4987-b4ce-41e2d429a03c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:37.079203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e9893-1512-4492-be30-966b02d71325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:37.079809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e9893-1512-4492-be30-966b02d71325 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:37.080089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e9893-1512-4492-be30-966b02d71325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:37.080360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e9893-1512-4492-be30-966b02d71325 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:37.081022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e9893-1512-4492-be30-966b02d71325 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:37.082978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e9893-1512-4492-be30-966b02d71325 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:37.111979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 939/1877] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:37 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:37.116350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb773a0-1956-4bd8-a628-021e03693c75 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:37.127818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb773a0-1956-4bd8-a628-021e03693c75 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:37.189416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998c3a3-984a-4809-a6fd-c475005d193d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:37.190233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c3a3-984a-4809-a6fd-c475005d193d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:37.190630 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c3a3-984a-4809-a6fd-c475005d193d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:37.190963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c3a3-984a-4809-a6fd-c475005d193d admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:37.191757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998c3a3-984a-4809-a6fd-c475005d193d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:37.194771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b998c3a3-984a-4809-a6fd-c475005d193d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:37.221522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 939/1878] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:37 2022] POST /identity/v3/users => generated 295 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:37.223147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e9893-1512-4492-be30-966b02d71325 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:37.233424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e9893-1512-4492-be30-966b02d71325 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:37.240922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dba95-0583-4f64-8c83-61c063d8daa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:37.241528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dba95-0583-4f64-8c83-61c063d8daa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:37.242194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dba95-0583-4f64-8c83-61c063d8daa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:37.242194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599dba95-0583-4f64-8c83-61c063d8daa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:37.242922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dba95-0583-4f64-8c83-61c063d8daa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:37.244518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599dba95-0583-4f64-8c83-61c063d8daa1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:37.260969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 940/1879] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:37.262194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998c3a3-984a-4809-a6fd-c475005d193d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:37.272384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998c3a3-984a-4809-a6fd-c475005d193d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:37.278095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781526c-b023-45a9-9757-adc0d2d33acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:37.278765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781526c-b023-45a9-9757-adc0d2d33acb admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:37.279042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781526c-b023-45a9-9757-adc0d2d33acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:37.279313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781526c-b023-45a9-9757-adc0d2d33acb admin admin] PATH_INFO: `/v3/projects/12b40bf4d234457aae486730758d50ec/users/dd2fc33f04c848aba970da169d356b31/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:37.295428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:37.298697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781526c-b023-45a9-9757-adc0d2d33acb admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b40bf4d234457aae486730758d50ec, user_id=dd2fc33f04c848aba970da169d356b31, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:37.301996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e781526c-b023-45a9-9757-adc0d2d33acb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:37.315383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 940/1880] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:37 2022] PUT /identity/v3/projects/12b40bf4d234457aae486730758d50ec/users/dd2fc33f04c848aba970da169d356b31/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:17:37.319617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dba95-0583-4f64-8c83-61c063d8daa1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:37.329645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599dba95-0583-4f64-8c83-61c063d8daa1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:37.373618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:37.390251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be163f29-bc8d-4a27-bc92-334010d50a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:37.390863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be163f29-bc8d-4a27-bc92-334010d50a08 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:37.391140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be163f29-bc8d-4a27-bc92-334010d50a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:37.391408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be163f29-bc8d-4a27-bc92-334010d50a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:37.392064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be163f29-bc8d-4a27-bc92-334010d50a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:37.393735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be163f29-bc8d-4a27-bc92-334010d50a08 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:37.403447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 941/1881] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:37.407713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781526c-b023-45a9-9757-adc0d2d33acb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:37.418622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781526c-b023-45a9-9757-adc0d2d33acb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:37.423496 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:37.424152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:37.424424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:37.424688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] PATH_INFO: `/v3/projects/12b40bf4d234457aae486730758d50ec/users/dd2fc33f04c848aba970da169d356b31/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:37.426664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b40bf4d234457aae486730758d50ec, user_id=dd2fc33f04c848aba970da169d356b31, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:37.429288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:37.441015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 941/1882] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:37 2022] PUT /identity/v3/projects/12b40bf4d234457aae486730758d50ec/users/dd2fc33f04c848aba970da169d356b31/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:37.452434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4854-2133-4b33-97d4-63d0229d8cb6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:37.452924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4854-2133-4b33-97d4-63d0229d8cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:37.452924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862d4854-2133-4b33-97d4-63d0229d8cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:37.482126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-862d4854-2133-4b33-97d4-63d0229d8cb6 None None] MFA Rules not processed for user `dd2fc33f04c848aba970da169d356b31`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:37.483325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-862d4854-2133-4b33-97d4-63d0229d8cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:37.553156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 942/1883] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:17:37.564936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:37.566957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:37.634692 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906a1ec-d244-4989-9666-9d3ceea57a38 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:37.638734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0b382-512b-493c-b605-b4f91f282146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:37.639329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0b382-512b-493c-b605-b4f91f282146 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:37.639609 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0b382-512b-493c-b605-b4f91f282146 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:37.639875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0b382-512b-493c-b605-b4f91f282146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:37.696777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0b382-512b-493c-b605-b4f91f282146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:37.698232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd0b382-512b-493c-b605-b4f91f282146 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:37.707502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 942/1884] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:43.944630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d4854-2133-4b33-97d4-63d0229d8cb6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:43.946376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-862d4854-2133-4b33-97d4-63d0229d8cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:43.959891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862d4854-2133-4b33-97d4-63d0229d8cb6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:43.965837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:43.966453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:43.966723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:43.966991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:43.969059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:44.037290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:44.038811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:44.048530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 943/1885] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:44.868684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0b382-512b-493c-b605-b4f91f282146 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:44.877906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0b382-512b-493c-b605-b4f91f282146 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:44.882609 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053c31e-95db-4dca-8786-7a5988c43c50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:44.883234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053c31e-95db-4dca-8786-7a5988c43c50 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:44.883512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053c31e-95db-4dca-8786-7a5988c43c50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:44.883841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053c31e-95db-4dca-8786-7a5988c43c50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:44.886040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c053c31e-95db-4dca-8786-7a5988c43c50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:44.954439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053c31e-95db-4dca-8786-7a5988c43c50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:44.955916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c053c31e-95db-4dca-8786-7a5988c43c50 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:44.967121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 943/1886] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:44 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:45.173522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:45.175091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:45.242114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d11aa-2bbc-47e7-aa96-9c6c590092c7 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:45.247877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:45.248511 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:45.248785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:45.249048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] PATH_INFO: `/v3/users/d1d1e7704cd34dfaa93b0b53659e6930` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:45.250277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d1e7704cd34dfaa93b0b53659e6930)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:45.252206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:45.253731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:45.314468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d1e7704cd34dfaa93b0b53659e6930'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:45.321423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d1e7704cd34dfaa93b0b53659e6930'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:45.322186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d1e7704cd34dfaa93b0b53659e6930'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:45.332859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d1e7704cd34dfaa93b0b53659e6930'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:45.345816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d1e7704cd34dfaa93b0b53659e6930'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:45.352189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 944/1887] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:17:45 2022] DELETE /identity/v3/users/d1d1e7704cd34dfaa93b0b53659e6930 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:45.357087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e7fa9-84a6-4463-8c66-999e028c7fa3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:45.357365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e7fa9-84a6-4463-8c66-999e028c7fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:45.357844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7e7fa9-84a6-4463-8c66-999e028c7fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:45.378076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6c7e7fa9-84a6-4463-8c66-999e028c7fa3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:45.379992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c7e7fa9-84a6-4463-8c66-999e028c7fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:45.405118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:45.448725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 944/1888] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:45.459375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:45.461367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:45.526345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ddd59-1a1a-4d44-8431-bc1145f143a4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:45.530203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f050da0-051b-4f02-9c55-9081b920b0f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:45.530820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f050da0-051b-4f02-9c55-9081b920b0f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:45.531091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f050da0-051b-4f02-9c55-9081b920b0f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:45.531346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f050da0-051b-4f02-9c55-9081b920b0f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:45.540282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f050da0-051b-4f02-9c55-9081b920b0f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:45.541740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f050da0-051b-4f02-9c55-9081b920b0f2 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:45.550868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 945/1889] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:46.004072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e7fa9-84a6-4463-8c66-999e028c7fa3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:46.005570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c7e7fa9-84a6-4463-8c66-999e028c7fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:46.018385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7e7fa9-84a6-4463-8c66-999e028c7fa3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:46.023289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:46.023944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:46.024234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:46.024552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] PATH_INFO: `/v3/projects/742d179f3db94f228de2efa4b547510e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:46.025742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] RBAC: Authorizing `identity:delete_project(project_id=742d179f3db94f228de2efa4b547510e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:46.027662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:46.078453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '742d179f3db94f228de2efa4b547510e', 'request_id': 'req-2b9713b6-c720-407e-afb0-4c2c27423318'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.084238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '742d179f3db94f228de2efa4b547510e', 'request_id': 'req-2b9713b6-c720-407e-afb0-4c2c27423318'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.090708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '742d179f3db94f228de2efa4b547510e', 'request_id': 'req-2b9713b6-c720-407e-afb0-4c2c27423318'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.105476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] The token cache is being invalidate because project 742d179f3db94f228de2efa4b547510e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:17:46.105813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.106461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.108051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 945/1890] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:17:46 2022] DELETE /identity/v3/projects/742d179f3db94f228de2efa4b547510e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:46.112302 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f050da0-051b-4f02-9c55-9081b920b0f2 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:46.122181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f050da0-051b-4f02-9c55-9081b920b0f2 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:46.179067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:46.179842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:46.179842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:46.180138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] PATH_INFO: `/v3/users/7f61a0e16c53452391f9e03467222664` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:46.181360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f61a0e16c53452391f9e03467222664)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:46.183347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:46.206490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:46.237412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f61a0e16c53452391f9e03467222664'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.245034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f61a0e16c53452391f9e03467222664'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.245754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f61a0e16c53452391f9e03467222664'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.267165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f61a0e16c53452391f9e03467222664'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.279750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e1ee442b-b4ac-428d-b265-c01b4bf79f58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f61a0e16c53452391f9e03467222664'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.285770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 946/1891] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:17:46 2022] DELETE /identity/v3/users/7f61a0e16c53452391f9e03467222664 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:46.584061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:46.585590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:46.599532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9713b6-c720-407e-afb0-4c2c27423318 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:46.657337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:46.658110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:46.658431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:46.658750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] PATH_INFO: `/v3/projects/9fe718343f7749f8a200db7425d80d2a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:46.659941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fe718343f7749f8a200db7425d80d2a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:46.661894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:46.712330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fe718343f7749f8a200db7425d80d2a', 'request_id': 'req-43136998-09a0-46ce-93f6-675ca05fe020'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.714258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:46.718673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fe718343f7749f8a200db7425d80d2a', 'request_id': 'req-43136998-09a0-46ce-93f6-675ca05fe020'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.725372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fe718343f7749f8a200db7425d80d2a', 'request_id': 'req-43136998-09a0-46ce-93f6-675ca05fe020'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.738390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] The token cache is being invalidate because project 9fe718343f7749f8a200db7425d80d2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:17:46.738669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.739268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-43136998-09a0-46ce-93f6-675ca05fe020 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:46.740809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 946/1892] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:17:46 2022] DELETE /identity/v3/projects/9fe718343f7749f8a200db7425d80d2a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:46.746613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b31857-ea44-421e-9810-309576d4399b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:46.747030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b31857-ea44-421e-9810-309576d4399b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:46.747335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b31857-ea44-421e-9810-309576d4399b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:46.767788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e7b31857-ea44-421e-9810-309576d4399b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:46.769180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e7b31857-ea44-421e-9810-309576d4399b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:46.829010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 947/1893] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:17:46.837213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb78fe-878c-4acf-8aa1-199b8e5cb3d9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:46.837492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb78fe-878c-4acf-8aa1-199b8e5cb3d9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:46.837759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb78fe-878c-4acf-8aa1-199b8e5cb3d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:46.860900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-40bb78fe-878c-4acf-8aa1-199b8e5cb3d9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:46.862261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-40bb78fe-878c-4acf-8aa1-199b8e5cb3d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:46.871166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 947/1894] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:17:46.875339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b31857-ea44-421e-9810-309576d4399b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:46.885022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b31857-ea44-421e-9810-309576d4399b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:46.890015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:46.890612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:46.890883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:46.891153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:46.891794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:17:46.892119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:46.893833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:46.905773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 948/1895] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:17:46.910271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb78fe-878c-4acf-8aa1-199b8e5cb3d9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:46.920009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb78fe-878c-4acf-8aa1-199b8e5cb3d9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:46.925732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09b39f-11d8-443f-9029-acd313b0b6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:46.926315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09b39f-11d8-443f-9029-acd313b0b6da admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:46.926582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09b39f-11d8-443f-9029-acd313b0b6da admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:46.926826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09b39f-11d8-443f-9029-acd313b0b6da admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:46.927455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09b39f-11d8-443f-9029-acd313b0b6da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:46.929357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09b39f-11d8-443f-9029-acd313b0b6da admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:46.960129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 948/1896] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:46 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:46.968460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:46.982434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbec14-445e-4ab5-92f5-6b032b13d3a5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.043398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0c4b4-247e-4f00-a4f7-fe379d84b232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.044014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0c4b4-247e-4f00-a4f7-fe379d84b232 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.044271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0c4b4-247e-4f00-a4f7-fe379d84b232 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.044546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e0c4b4-247e-4f00-a4f7-fe379d84b232 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.045266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0c4b4-247e-4f00-a4f7-fe379d84b232 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.047423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e0c4b4-247e-4f00-a4f7-fe379d84b232 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.070156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 949/1897] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:46 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:47.074610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09b39f-11d8-443f-9029-acd313b0b6da admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.085210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09b39f-11d8-443f-9029-acd313b0b6da admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.090401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcff062-a7d3-4cbe-a1b4-059d801498fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.090998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff062-a7d3-4cbe-a1b4-059d801498fd admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.091271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff062-a7d3-4cbe-a1b4-059d801498fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.091536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcff062-a7d3-4cbe-a1b4-059d801498fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.092177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcff062-a7d3-4cbe-a1b4-059d801498fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.093813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcff062-a7d3-4cbe-a1b4-059d801498fd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.104537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 949/1898] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:47.108464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0c4b4-247e-4f00-a4f7-fe379d84b232 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.118990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e0c4b4-247e-4f00-a4f7-fe379d84b232 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.124041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab5ba25-4889-46d3-967b-51beae7d5569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.124636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5ba25-4889-46d3-967b-51beae7d5569 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.124911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5ba25-4889-46d3-967b-51beae7d5569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.125201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab5ba25-4889-46d3-967b-51beae7d5569 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.125866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5ba25-4889-46d3-967b-51beae7d5569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.127459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab5ba25-4889-46d3-967b-51beae7d5569 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.138477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 950/1899] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:47.142771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcff062-a7d3-4cbe-a1b4-059d801498fd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.153125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcff062-a7d3-4cbe-a1b4-059d801498fd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.157953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7628e-b6b2-4fad-a0f3-3350a02d0f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.158591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7628e-b6b2-4fad-a0f3-3350a02d0f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.158865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7628e-b6b2-4fad-a0f3-3350a02d0f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.159134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd7628e-b6b2-4fad-a0f3-3350a02d0f1a admin admin] PATH_INFO: `/v3/projects/e0bf8d8baf034324bd9cdfae771a1fb3/users/b2f0f0ad39614af1bcf4bb97f44394ba/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.174288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7628e-b6b2-4fad-a0f3-3350a02d0f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bf8d8baf034324bd9cdfae771a1fb3, user_id=b2f0f0ad39614af1bcf4bb97f44394ba, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.176994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd7628e-b6b2-4fad-a0f3-3350a02d0f1a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.196447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 950/1900] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:47 2022] PUT /identity/v3/projects/e0bf8d8baf034324bd9cdfae771a1fb3/users/b2f0f0ad39614af1bcf4bb97f44394ba/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:47.213477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5d6f-1919-43ea-80f7-47d78a833859 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.214075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5d6f-1919-43ea-80f7-47d78a833859 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.214075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e5d6f-1919-43ea-80f7-47d78a833859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.227638 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:47.251510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-580e5d6f-1919-43ea-80f7-47d78a833859 None None] MFA Rules not processed for user `b2f0f0ad39614af1bcf4bb97f44394ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:47.252713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-580e5d6f-1919-43ea-80f7-47d78a833859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:47.335206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 951/1901] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:47.340002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7628e-b6b2-4fad-a0f3-3350a02d0f1a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.409578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd7628e-b6b2-4fad-a0f3-3350a02d0f1a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.414624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9da2-7ec0-40df-9c57-d0eff393831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.415240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9da2-7ec0-40df-9c57-d0eff393831c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.415514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9da2-7ec0-40df-9c57-d0eff393831c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.415779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf9da2-7ec0-40df-9c57-d0eff393831c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.416412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf9da2-7ec0-40df-9c57-d0eff393831c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.418356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf9da2-7ec0-40df-9c57-d0eff393831c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.451124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 951/1902] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:47 2022] POST /identity/v3/projects => generated 387 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:47.452201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e5d6f-1919-43ea-80f7-47d78a833859 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.463835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e5d6f-1919-43ea-80f7-47d78a833859 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.520544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3273b6-440e-41b2-8539-6dad57486219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.521190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3273b6-440e-41b2-8539-6dad57486219 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.521467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3273b6-440e-41b2-8539-6dad57486219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.521761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3273b6-440e-41b2-8539-6dad57486219 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.522389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3273b6-440e-41b2-8539-6dad57486219 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.524505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3273b6-440e-41b2-8539-6dad57486219 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.547298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 952/1903] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:47 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:17:47.551416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9da2-7ec0-40df-9c57-d0eff393831c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.583783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf9da2-7ec0-40df-9c57-d0eff393831c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.594327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.595342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.595568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.595839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.596496 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.598639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.617099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 952/1904] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:47.625132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3273b6-440e-41b2-8539-6dad57486219 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.632120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.633980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:47.641684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3273b6-440e-41b2-8539-6dad57486219 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.653315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb5f7-2d91-4c8c-93bc-715e5733927a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.653965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb5f7-2d91-4c8c-93bc-715e5733927a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.654238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb5f7-2d91-4c8c-93bc-715e5733927a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.654500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7eb5f7-2d91-4c8c-93bc-715e5733927a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.655141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eb5f7-2d91-4c8c-93bc-715e5733927a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.656740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7eb5f7-2d91-4c8c-93bc-715e5733927a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.667492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 953/1905] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:47.671530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb5f7-2d91-4c8c-93bc-715e5733927a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.681239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7eb5f7-2d91-4c8c-93bc-715e5733927a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.685942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f97eba-1eef-47ee-a856-bb9dc909989a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.686646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97eba-1eef-47ee-a856-bb9dc909989a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.686922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97eba-1eef-47ee-a856-bb9dc909989a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.687189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f97eba-1eef-47ee-a856-bb9dc909989a admin admin] PATH_INFO: `/v3/projects/d300498e17aa45dcbd164221dbec880c/users/481ac6f1439240c18fdfc968b5bd2240/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.694554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec36fad-517b-4098-b315-8b4f5d75979e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.698199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.698810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.699081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.699357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.701117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:47.702021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f97eba-1eef-47ee-a856-bb9dc909989a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d300498e17aa45dcbd164221dbec880c, user_id=481ac6f1439240c18fdfc968b5bd2240, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.704644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f97eba-1eef-47ee-a856-bb9dc909989a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.706724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:47.727499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 954/1906] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:47 2022] PUT /identity/v3/projects/d300498e17aa45dcbd164221dbec880c/users/481ac6f1439240c18fdfc968b5bd2240/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:17:47.743263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd8e89-93bb-4471-9ba6-159879ad3517 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.743642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd8e89-93bb-4471-9ba6-159879ad3517 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.744003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd8e89-93bb-4471-9ba6-159879ad3517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.768453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.769927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.771513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-adcd8e89-93bb-4471-9ba6-159879ad3517 None None] MFA Rules not processed for user `481ac6f1439240c18fdfc968b5bd2240`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:47.772705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-adcd8e89-93bb-4471-9ba6-159879ad3517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:47.779563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 953/1907] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:47.816983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:47.848442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.849941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:47.854348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 955/1908] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:47.855285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd8e89-93bb-4471-9ba6-159879ad3517 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:47.876655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3ba72c-a4bd-408c-af95-c2be781aedad service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.923662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd8e89-93bb-4471-9ba6-159879ad3517 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:47.930248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4184137-d2a5-4cd8-8f6f-557670bf2330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.930999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4184137-d2a5-4cd8-8f6f-557670bf2330 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.931282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4184137-d2a5-4cd8-8f6f-557670bf2330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.931668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4184137-d2a5-4cd8-8f6f-557670bf2330 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.932326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4184137-d2a5-4cd8-8f6f-557670bf2330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.934428 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4184137-d2a5-4cd8-8f6f-557670bf2330 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.949914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:47.950724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:47.951440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:47.951727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] PATH_INFO: `/v3/users/1fb013e26c664e8f8ef302accd8baee5` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:47.953070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fb013e26c664e8f8ef302accd8baee5)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:47.955047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:47.963683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 956/1909] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:47 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:47.967681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4184137-d2a5-4cd8-8f6f-557670bf2330 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.004579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fb013e26c664e8f8ef302accd8baee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.011826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fb013e26c664e8f8ef302accd8baee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.012513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fb013e26c664e8f8ef302accd8baee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.020781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fb013e26c664e8f8ef302accd8baee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.030294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4184137-d2a5-4cd8-8f6f-557670bf2330 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.033474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c9a3a6b3-3025-4e5b-a035-f5a54beeb810 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fb013e26c664e8f8ef302accd8baee5'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.035318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.035965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.036256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.036455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.037183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.039191 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.039782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 954/1910] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:17:47 2022] DELETE /identity/v3/users/1fb013e26c664e8f8ef302accd8baee5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:17:48.044312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166ec07-3124-4b62-b9a1-23b87b04b6f7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.044591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166ec07-3124-4b62-b9a1-23b87b04b6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.044985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166ec07-3124-4b62-b9a1-23b87b04b6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.064442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 957/1911] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:47 2022] POST /identity/v3/users => generated 300 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:17:48.066373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d166ec07-3124-4b62-b9a1-23b87b04b6f7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:48.067566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d166ec07-3124-4b62-b9a1-23b87b04b6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:48.068364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.069969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:48.080581 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:48.081841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6673eadc-01d1-4fd6-99b7-a2259a67ce4e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.132364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 955/1912] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:17:48.132604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:48.142410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166ec07-3124-4b62-b9a1-23b87b04b6f7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.143743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3a60c-f0f6-4ebd-acd4-93c6b1343884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.143926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d166ec07-3124-4b62-b9a1-23b87b04b6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:48.144523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3a60c-f0f6-4ebd-acd4-93c6b1343884 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.144882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3a60c-f0f6-4ebd-acd4-93c6b1343884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.145053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be3a60c-f0f6-4ebd-acd4-93c6b1343884 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.145733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3a60c-f0f6-4ebd-acd4-93c6b1343884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.147352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be3a60c-f0f6-4ebd-acd4-93c6b1343884 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.160604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 958/1913] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:48.161551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3a60c-f0f6-4ebd-acd4-93c6b1343884 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.171040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be3a60c-f0f6-4ebd-acd4-93c6b1343884 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.175808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51416b64-2ec4-448d-a566-27697542d4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.176434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51416b64-2ec4-448d-a566-27697542d4cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.176705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51416b64-2ec4-448d-a566-27697542d4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.176973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51416b64-2ec4-448d-a566-27697542d4cc admin admin] PATH_INFO: `/v3/projects/00215c4d262949c0a4e9f81647504104/users/d4985d664d6441f8893e12140e19329f/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.191727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51416b64-2ec4-448d-a566-27697542d4cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=00215c4d262949c0a4e9f81647504104, user_id=d4985d664d6441f8893e12140e19329f, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.194446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51416b64-2ec4-448d-a566-27697542d4cc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.201034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166ec07-3124-4b62-b9a1-23b87b04b6f7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.204583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bd118-8255-4eaf-ae10-f3125133fc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.205180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bd118-8255-4eaf-ae10-f3125133fc7c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.205457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bd118-8255-4eaf-ae10-f3125133fc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.205751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882bd118-8255-4eaf-ae10-f3125133fc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.208217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 959/1914] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:48 2022] PUT /identity/v3/projects/00215c4d262949c0a4e9f81647504104/users/d4985d664d6441f8893e12140e19329f/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:48.212394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51416b64-2ec4-448d-a566-27697542d4cc admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.215055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bd118-8255-4eaf-ae10-f3125133fc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.216481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882bd118-8255-4eaf-ae10-f3125133fc7c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.226235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 956/1915] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:17:48.275009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51416b64-2ec4-448d-a566-27697542d4cc admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.281067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db45a39-f524-40a4-9e69-67c80c665933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.281842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45a39-f524-40a4-9e69-67c80c665933 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.282151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45a39-f524-40a4-9e69-67c80c665933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.282443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db45a39-f524-40a4-9e69-67c80c665933 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.283153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db45a39-f524-40a4-9e69-67c80c665933 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.284821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db45a39-f524-40a4-9e69-67c80c665933 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.295913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 960/1916] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:48.300217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bd118-8255-4eaf-ae10-f3125133fc7c service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.310002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882bd118-8255-4eaf-ae10-f3125133fc7c service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.314851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.315526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.315800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.316072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] PATH_INFO: `/v3/projects/00215c4d262949c0a4e9f81647504104/users/d4985d664d6441f8893e12140e19329f/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.318070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00215c4d262949c0a4e9f81647504104, user_id=d4985d664d6441f8893e12140e19329f, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.320692 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.342808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 957/1917] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:48 2022] PUT /identity/v3/projects/00215c4d262949c0a4e9f81647504104/users/d4985d664d6441f8893e12140e19329f/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:48.344542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1633b-62a0-47eb-83c3-f5ec3c5589fb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.344827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1633b-62a0-47eb-83c3-f5ec3c5589fb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.345098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1633b-62a0-47eb-83c3-f5ec3c5589fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.388926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ddd1633b-62a0-47eb-83c3-f5ec3c5589fb None None] MFA Rules not processed for user `d4985d664d6441f8893e12140e19329f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:48.390189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ddd1633b-62a0-47eb-83c3-f5ec3c5589fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:48.458117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 961/1918] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:48.469151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.470839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:48.513229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1633b-62a0-47eb-83c3-f5ec3c5589fb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.515194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ddd1633b-62a0-47eb-83c3-f5ec3c5589fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:48.546562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11be1e18-aca1-4b4d-af42-ba7d68cae424 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.550332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239422-5e30-4768-a22f-f9e3e068ead6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.550954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239422-5e30-4768-a22f-f9e3e068ead6 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.551215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239422-5e30-4768-a22f-f9e3e068ead6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.551476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00239422-5e30-4768-a22f-f9e3e068ead6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.561870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00239422-5e30-4768-a22f-f9e3e068ead6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.563295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00239422-5e30-4768-a22f-f9e3e068ead6 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.572768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 958/1919] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:17:48.575480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:48.592075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1633b-62a0-47eb-83c3-f5ec3c5589fb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.596691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.597718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.597995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.598259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] PATH_INFO: `/v3/projects/b5bd1ac9808a4eb79f6b48bdf93c24c9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.599363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5bd1ac9808a4eb79f6b48bdf93c24c9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.601266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.662626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5bd1ac9808a4eb79f6b48bdf93c24c9', 'request_id': 'req-e108752e-033e-4439-8fbe-b9087e8fd751'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.662953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5bd1ac9808a4eb79f6b48bdf93c24c9', 'request_id': 'req-e108752e-033e-4439-8fbe-b9087e8fd751'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.670676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5bd1ac9808a4eb79f6b48bdf93c24c9', 'request_id': 'req-e108752e-033e-4439-8fbe-b9087e8fd751'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.689594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] The token cache is being invalidate because project b5bd1ac9808a4eb79f6b48bdf93c24c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:17:48.690259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.691339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.700182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239422-5e30-4768-a22f-f9e3e068ead6 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:48.701712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 962/1920] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:17:48 2022] DELETE /identity/v3/projects/b5bd1ac9808a4eb79f6b48bdf93c24c9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:48.718241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00239422-5e30-4768-a22f-f9e3e068ead6 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:48.786846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:48.795612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:48.796262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:48.796537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:48.796808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] PATH_INFO: `/v3/users/d198e465168145a58103dffcd3d5e555` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:48.798004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d198e465168145a58103dffcd3d5e555)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:48.800095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:48.848307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd198e465168145a58103dffcd3d5e555'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.855226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd198e465168145a58103dffcd3d5e555'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.857837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd198e465168145a58103dffcd3d5e555'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.866323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd198e465168145a58103dffcd3d5e555'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.878422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd198e465168145a58103dffcd3d5e555'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:48.884447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 959/1921] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:17:48 2022] DELETE /identity/v3/users/d198e465168145a58103dffcd3d5e555 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:49.154696 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.155720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:49.178902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e108752e-033e-4439-8fbe-b9087e8fd751 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.189262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.191516 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:49.242038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.242732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.243032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.243318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] PATH_INFO: `/v3/projects/60f1acdb72b14a9bb7762ab6254f04ec` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.244423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60f1acdb72b14a9bb7762ab6254f04ec)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.246374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:49.250617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6628300-5baf-4596-87c4-84c8640b6a8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.254638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abbccb9-07ab-497d-b8d5-804b177a1a62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.254939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbccb9-07ab-497d-b8d5-804b177a1a62 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.255209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbccb9-07ab-497d-b8d5-804b177a1a62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.255505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abbccb9-07ab-497d-b8d5-804b177a1a62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.257253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1abbccb9-07ab-497d-b8d5-804b177a1a62 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:49.294078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60f1acdb72b14a9bb7762ab6254f04ec', 'request_id': 'req-06452e6b-9139-4425-bada-02d1682f3142'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:49.299569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60f1acdb72b14a9bb7762ab6254f04ec', 'request_id': 'req-06452e6b-9139-4425-bada-02d1682f3142'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:49.305934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60f1acdb72b14a9bb7762ab6254f04ec', 'request_id': 'req-06452e6b-9139-4425-bada-02d1682f3142'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:49.316491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbccb9-07ab-497d-b8d5-804b177a1a62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.318133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abbccb9-07ab-497d-b8d5-804b177a1a62 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:49.320508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] The token cache is being invalidate because project 60f1acdb72b14a9bb7762ab6254f04ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:17:49.320816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:49.321498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-06452e6b-9139-4425-bada-02d1682f3142 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:49.323273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 963/1922] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:17:49 2022] DELETE /identity/v3/projects/60f1acdb72b14a9bb7762ab6254f04ec => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:49.328870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 960/1923] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:17:49.329300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cca4-08c7-4c2e-83c5-77283e0acd9b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.329583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cca4-08c7-4c2e-83c5-77283e0acd9b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.329887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164cca4-08c7-4c2e-83c5-77283e0acd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.350031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e164cca4-08c7-4c2e-83c5-77283e0acd9b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:49.351234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e164cca4-08c7-4c2e-83c5-77283e0acd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:49.355554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:49.416326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 964/1924] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:17:49.424173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c701-9fe5-4f07-ba98-68fd36ef0c29 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.424436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c701-9fe5-4f07-ba98-68fd36ef0c29 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.424729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243c701-9fe5-4f07-ba98-68fd36ef0c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.445855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a243c701-9fe5-4f07-ba98-68fd36ef0c29 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:49.447230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a243c701-9fe5-4f07-ba98-68fd36ef0c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:49.455912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 961/1925] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:49.460065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164cca4-08c7-4c2e-83c5-77283e0acd9b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.470288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164cca4-08c7-4c2e-83c5-77283e0acd9b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.475516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.476304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.476592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.476860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.477537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:17:49.477875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.479485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:49.494686 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 965/1926] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:17:49.499196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c701-9fe5-4f07-ba98-68fd36ef0c29 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.510562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243c701-9fe5-4f07-ba98-68fd36ef0c29 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.515689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246904c5-b314-4232-a166-5de05c00d269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.516426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246904c5-b314-4232-a166-5de05c00d269 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.516706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246904c5-b314-4232-a166-5de05c00d269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.517004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246904c5-b314-4232-a166-5de05c00d269 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.517685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246904c5-b314-4232-a166-5de05c00d269 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.519686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246904c5-b314-4232-a166-5de05c00d269 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:49.548231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:49.557358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 962/1927] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:49 2022] POST /identity/v3/projects => generated 389 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:17:49.562209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.574296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93e1888-7931-45cc-841a-ca0344913651 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.646465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc6316-584b-418d-93a6-94cc801f7330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.647117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc6316-584b-418d-93a6-94cc801f7330 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.647391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc6316-584b-418d-93a6-94cc801f7330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.647656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc6316-584b-418d-93a6-94cc801f7330 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.648306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc6316-584b-418d-93a6-94cc801f7330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.650419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc6316-584b-418d-93a6-94cc801f7330 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:49.678568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 966/1928] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:49 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:49.680216 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246904c5-b314-4232-a166-5de05c00d269 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.690972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246904c5-b314-4232-a166-5de05c00d269 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.696173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e841cc-0ae8-4181-98fb-f4179b77db1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.696817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e841cc-0ae8-4181-98fb-f4179b77db1d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.697206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e841cc-0ae8-4181-98fb-f4179b77db1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.697489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e841cc-0ae8-4181-98fb-f4179b77db1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.698158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e841cc-0ae8-4181-98fb-f4179b77db1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.699778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e841cc-0ae8-4181-98fb-f4179b77db1d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:49.709864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 963/1929] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:17:49.714122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc6316-584b-418d-93a6-94cc801f7330 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.724694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc6316-584b-418d-93a6-94cc801f7330 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.729681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef913bfa-5f31-4804-aa43-4ca629b460b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.730173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef913bfa-5f31-4804-aa43-4ca629b460b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.730447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef913bfa-5f31-4804-aa43-4ca629b460b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.730712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef913bfa-5f31-4804-aa43-4ca629b460b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.731528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef913bfa-5f31-4804-aa43-4ca629b460b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.733122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef913bfa-5f31-4804-aa43-4ca629b460b0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:49.745543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 967/1930] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:49.749804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e841cc-0ae8-4181-98fb-f4179b77db1d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.759915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e841cc-0ae8-4181-98fb-f4179b77db1d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.764506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a1d34-80ea-4810-93bf-dc2a4523c450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.765151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1d34-80ea-4810-93bf-dc2a4523c450 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.765450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1d34-80ea-4810-93bf-dc2a4523c450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.765753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a1d34-80ea-4810-93bf-dc2a4523c450 admin admin] PATH_INFO: `/v3/projects/dd4e4424f09a4e5586b100dbc6e066e5/users/96f06b7f40d747269843ec73ebc66d74/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.782878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a1d34-80ea-4810-93bf-dc2a4523c450 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4e4424f09a4e5586b100dbc6e066e5, user_id=96f06b7f40d747269843ec73ebc66d74, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.785684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a1d34-80ea-4810-93bf-dc2a4523c450 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:49.800187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 964/1931] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:49 2022] PUT /identity/v3/projects/dd4e4424f09a4e5586b100dbc6e066e5/users/96f06b7f40d747269843ec73ebc66d74/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:49.810985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9107137-d76d-4d6a-8a75-3342b0ef9e97 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.811292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9107137-d76d-4d6a-8a75-3342b0ef9e97 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.811540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9107137-d76d-4d6a-8a75-3342b0ef9e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.839119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b9107137-d76d-4d6a-8a75-3342b0ef9e97 None None] MFA Rules not processed for user `96f06b7f40d747269843ec73ebc66d74`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:49.840383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b9107137-d76d-4d6a-8a75-3342b0ef9e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:49.907499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 968/1932] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:49.912357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a1d34-80ea-4810-93bf-dc2a4523c450 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:49.982621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a1d34-80ea-4810-93bf-dc2a4523c450 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:49.987352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010bf67-339b-4c00-a5b8-91c82b70388d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:49.987964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf67-339b-4c00-a5b8-91c82b70388d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:49.988236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf67-339b-4c00-a5b8-91c82b70388d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:49.988635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010bf67-339b-4c00-a5b8-91c82b70388d admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:49.989321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010bf67-339b-4c00-a5b8-91c82b70388d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:49.991311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010bf67-339b-4c00-a5b8-91c82b70388d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:50.020177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 965/1933] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:49 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:17:50.025031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9107137-d76d-4d6a-8a75-3342b0ef9e97 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:50.033221 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:50.038463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9107137-d76d-4d6a-8a75-3342b0ef9e97 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:50.094856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218ca1-a4ae-4ed2-bae6-5fdeb275bb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:50.095385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218ca1-a4ae-4ed2-bae6-5fdeb275bb93 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.095658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218ca1-a4ae-4ed2-bae6-5fdeb275bb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.095920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218ca1-a4ae-4ed2-bae6-5fdeb275bb93 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.096527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0218ca1-a4ae-4ed2-bae6-5fdeb275bb93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:50.098826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0218ca1-a4ae-4ed2-bae6-5fdeb275bb93 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:50.122141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 969/1934] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:50 2022] POST /identity/v3/users => generated 301 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:50.127544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010bf67-339b-4c00-a5b8-91c82b70388d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:50.137886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010bf67-339b-4c00-a5b8-91c82b70388d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:50.142729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affe4b4-bd5d-4cbe-b3b4-7e275ad52a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:50.143323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affe4b4-bd5d-4cbe-b3b4-7e275ad52a8f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.143629 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affe4b4-bd5d-4cbe-b3b4-7e275ad52a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.143896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7affe4b4-bd5d-4cbe-b3b4-7e275ad52a8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.144584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affe4b4-bd5d-4cbe-b3b4-7e275ad52a8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:50.146210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7affe4b4-bd5d-4cbe-b3b4-7e275ad52a8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:50.161790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 966/1935] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:17:50.167523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218ca1-a4ae-4ed2-bae6-5fdeb275bb93 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:50.177973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218ca1-a4ae-4ed2-bae6-5fdeb275bb93 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:50.183899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89b267-8dc1-4a8f-9906-5af4c69f9fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:50.184562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b267-8dc1-4a8f-9906-5af4c69f9fa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.184835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b267-8dc1-4a8f-9906-5af4c69f9fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.185085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b89b267-8dc1-4a8f-9906-5af4c69f9fa0 admin admin] PATH_INFO: `/v3/projects/70e5763a2a3043fcbc93f0702008254e/users/52f1f2a5af144e948b80d0dee0381874/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.200794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89b267-8dc1-4a8f-9906-5af4c69f9fa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e5763a2a3043fcbc93f0702008254e, user_id=52f1f2a5af144e948b80d0dee0381874, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:50.203515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b89b267-8dc1-4a8f-9906-5af4c69f9fa0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:50.218931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 970/1936] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:50 2022] PUT /identity/v3/projects/70e5763a2a3043fcbc93f0702008254e/users/52f1f2a5af144e948b80d0dee0381874/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:50.224122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affe4b4-bd5d-4cbe-b3b4-7e275ad52a8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:50.237248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7affe4b4-bd5d-4cbe-b3b4-7e275ad52a8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:50.295550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a017093-5194-47ae-96fb-d7f03231e932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:50.296155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a017093-5194-47ae-96fb-d7f03231e932 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.296425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a017093-5194-47ae-96fb-d7f03231e932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.296689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a017093-5194-47ae-96fb-d7f03231e932 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.297326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a017093-5194-47ae-96fb-d7f03231e932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:50.298956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a017093-5194-47ae-96fb-d7f03231e932 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:50.309063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 967/1937] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:50.313506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89b267-8dc1-4a8f-9906-5af4c69f9fa0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:50.324645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b89b267-8dc1-4a8f-9906-5af4c69f9fa0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:50.330395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:50.331054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.331333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.331593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] PATH_INFO: `/v3/projects/70e5763a2a3043fcbc93f0702008254e/users/52f1f2a5af144e948b80d0dee0381874/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.333802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=70e5763a2a3043fcbc93f0702008254e, user_id=52f1f2a5af144e948b80d0dee0381874, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:50.336488 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:50.347904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 971/1938] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:50 2022] PUT /identity/v3/projects/70e5763a2a3043fcbc93f0702008254e/users/52f1f2a5af144e948b80d0dee0381874/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:50.359327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42cb0-4347-4289-949e-e5e7872caddb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.359605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42cb0-4347-4289-949e-e5e7872caddb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.359867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b42cb0-4347-4289-949e-e5e7872caddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.389294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f7b42cb0-4347-4289-949e-e5e7872caddb None None] MFA Rules not processed for user `52f1f2a5af144e948b80d0dee0381874`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:50.390544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f7b42cb0-4347-4289-949e-e5e7872caddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:50.410499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:50.458992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 968/1939] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:50 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:50.470043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:50.472507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:50.541286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c7ca3c-68d1-468d-8cef-8b2af67876ba admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:50.545384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9b85e6-8684-4ee2-a9ee-321fd7941758 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:50.546046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b85e6-8684-4ee2-a9ee-321fd7941758 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.546348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b85e6-8684-4ee2-a9ee-321fd7941758 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.546593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9b85e6-8684-4ee2-a9ee-321fd7941758 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.556018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b85e6-8684-4ee2-a9ee-321fd7941758 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:50.557444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9b85e6-8684-4ee2-a9ee-321fd7941758 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:50.566868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 972/1940] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:50 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:50.708024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42cb0-4347-4289-949e-e5e7872caddb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:50.773167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b42cb0-4347-4289-949e-e5e7872caddb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:50.778203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:50.778840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.779119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.779390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] PATH_INFO: `/v3/users/96f06b7f40d747269843ec73ebc66d74` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.780560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96f06b7f40d747269843ec73ebc66d74)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:50.782521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:50.833874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96f06b7f40d747269843ec73ebc66d74'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:50.843207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96f06b7f40d747269843ec73ebc66d74'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:50.843889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96f06b7f40d747269843ec73ebc66d74'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:50.851885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96f06b7f40d747269843ec73ebc66d74'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:50.864508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96f06b7f40d747269843ec73ebc66d74'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:50.870259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 969/1941] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:17:50 2022] DELETE /identity/v3/users/96f06b7f40d747269843ec73ebc66d74 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:50.874729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e074-324b-4b05-bc09-601e58672fa2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:50.875002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e074-324b-4b05-bc09-601e58672fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:50.875266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2318e074-324b-4b05-bc09-601e58672fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:50.894120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2318e074-324b-4b05-bc09-601e58672fa2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:50.895297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2318e074-324b-4b05-bc09-601e58672fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:50.947774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:50.957599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 973/1942] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:50.969619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:50.971111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:51.016047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:51.033323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bdfe31-8d8f-4c85-ae6d-727461376af0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:51.036927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8024952-677c-4777-8ef5-01a07fe8021b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:51.037541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8024952-677c-4777-8ef5-01a07fe8021b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:51.037846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8024952-677c-4777-8ef5-01a07fe8021b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:51.038240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8024952-677c-4777-8ef5-01a07fe8021b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:51.047404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8024952-677c-4777-8ef5-01a07fe8021b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:51.048873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8024952-677c-4777-8ef5-01a07fe8021b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:51.058338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 970/1943] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:51.354030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e074-324b-4b05-bc09-601e58672fa2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:51.355772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2318e074-324b-4b05-bc09-601e58672fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:51.369498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2318e074-324b-4b05-bc09-601e58672fa2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:51.375867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:51.376891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:51.377172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:51.377441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] PATH_INFO: `/v3/projects/dd4e4424f09a4e5586b100dbc6e066e5` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:51.378652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4e4424f09a4e5586b100dbc6e066e5)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:51.380583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:51.433828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4e4424f09a4e5586b100dbc6e066e5', 'request_id': 'req-53cd8483-3952-403d-ac2c-50f5d9aa980c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.440998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4e4424f09a4e5586b100dbc6e066e5', 'request_id': 'req-53cd8483-3952-403d-ac2c-50f5d9aa980c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.448878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4e4424f09a4e5586b100dbc6e066e5', 'request_id': 'req-53cd8483-3952-403d-ac2c-50f5d9aa980c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.466281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] The token cache is being invalidate because project dd4e4424f09a4e5586b100dbc6e066e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:17:51.466563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.467344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.469094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 974/1944] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:17:51 2022] DELETE /identity/v3/projects/dd4e4424f09a4e5586b100dbc6e066e5 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:51.473589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8024952-677c-4777-8ef5-01a07fe8021b service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:51.484348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8024952-677c-4777-8ef5-01a07fe8021b service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:51.550142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:51.550795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:51.551076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:51.551354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] PATH_INFO: `/v3/users/52f1f2a5af144e948b80d0dee0381874` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:51.552506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] RBAC: Authorizing `identity:delete_user(user_id=52f1f2a5af144e948b80d0dee0381874)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:51.554474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:51.607578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52f1f2a5af144e948b80d0dee0381874'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.614416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52f1f2a5af144e948b80d0dee0381874'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.615071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52f1f2a5af144e948b80d0dee0381874'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.623729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52f1f2a5af144e948b80d0dee0381874'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.637429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-4d614fd5-3f67-4a1b-8019-963a330bbdae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52f1f2a5af144e948b80d0dee0381874'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:51.643374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 971/1945] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:17:51 2022] DELETE /identity/v3/users/52f1f2a5af144e948b80d0dee0381874 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:17:51.921618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:51.924934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:51.940513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cd8483-3952-403d-ac2c-50f5d9aa980c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:51.999606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.000231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.000504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.000771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] PATH_INFO: `/v3/projects/70e5763a2a3043fcbc93f0702008254e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.001919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70e5763a2a3043fcbc93f0702008254e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.003828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.057197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70e5763a2a3043fcbc93f0702008254e', 'request_id': 'req-a7e176b6-1288-4077-a64c-b852e8b3b4b0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:52.062971 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70e5763a2a3043fcbc93f0702008254e', 'request_id': 'req-a7e176b6-1288-4077-a64c-b852e8b3b4b0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:52.069882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70e5763a2a3043fcbc93f0702008254e', 'request_id': 'req-a7e176b6-1288-4077-a64c-b852e8b3b4b0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:52.084056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] The token cache is being invalidate because project 70e5763a2a3043fcbc93f0702008254e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:17:52.084399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:52.085178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a7e176b6-1288-4077-a64c-b852e8b3b4b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:17:52.086790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 975/1946] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:17:51 2022] DELETE /identity/v3/projects/70e5763a2a3043fcbc93f0702008254e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:52.091822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c40c-766b-44f4-838b-07c54f4ff5d1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.092174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c40c-766b-44f4-838b-07c54f4ff5d1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.092512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9c40c-766b-44f4-838b-07c54f4ff5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.112792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ebe9c40c-766b-44f4-838b-07c54f4ff5d1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:52.114030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ebe9c40c-766b-44f4-838b-07c54f4ff5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:52.166611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:52.177467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 972/1947] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:52.185480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c861-ad0d-49c7-aa77-b19d74e42d7d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.185792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c861-ad0d-49c7-aa77-b19d74e42d7d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.186067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2c861-ad0d-49c7-aa77-b19d74e42d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.206883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-37b2c861-ad0d-49c7-aa77-b19d74e42d7d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:52.208152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-37b2c861-ad0d-49c7-aa77-b19d74e42d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:52.216477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 976/1948] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:52.221105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9c40c-766b-44f4-838b-07c54f4ff5d1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.231844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9c40c-766b-44f4-838b-07c54f4ff5d1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.237949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.238561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.238834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.239103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.239753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:17:52.240050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.241671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.253654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 973/1949] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:17:52.258031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c861-ad0d-49c7-aa77-b19d74e42d7d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.267571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2c861-ad0d-49c7-aa77-b19d74e42d7d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.272623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec239f1-fdec-4592-bf59-ab9185d01ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.273223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec239f1-fdec-4592-bf59-ab9185d01ab1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.273499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec239f1-fdec-4592-bf59-ab9185d01ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.273801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec239f1-fdec-4592-bf59-ab9185d01ab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.274433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec239f1-fdec-4592-bf59-ab9185d01ab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.276375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec239f1-fdec-4592-bf59-ab9185d01ab1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.280899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:52.309047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 977/1950] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:52 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:52.313236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.323712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb3ea33-327a-45fe-9c33-2b3f23e914c7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.385850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cb94f-ec32-4e94-97fb-3a524a3783fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.386463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cb94f-ec32-4e94-97fb-3a524a3783fa admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.386733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cb94f-ec32-4e94-97fb-3a524a3783fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.387000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7cb94f-ec32-4e94-97fb-3a524a3783fa admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.387640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cb94f-ec32-4e94-97fb-3a524a3783fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.389578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7cb94f-ec32-4e94-97fb-3a524a3783fa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.412587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 974/1951] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:52 2022] POST /identity/v3/users => generated 305 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:52.416923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec239f1-fdec-4592-bf59-ab9185d01ab1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.427250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec239f1-fdec-4592-bf59-ab9185d01ab1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.432086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e59e4-24b7-496a-94a3-d287977894d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.432691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e59e4-24b7-496a-94a3-d287977894d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.432966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e59e4-24b7-496a-94a3-d287977894d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.433231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e59e4-24b7-496a-94a3-d287977894d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.433911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e59e4-24b7-496a-94a3-d287977894d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.435505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e59e4-24b7-496a-94a3-d287977894d8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.445942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 978/1952] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:52.449910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cb94f-ec32-4e94-97fb-3a524a3783fa admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.462003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7cb94f-ec32-4e94-97fb-3a524a3783fa admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.467608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeb4ac1-deca-4bf1-bfc0-0080a6b7ae5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.468208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb4ac1-deca-4bf1-bfc0-0080a6b7ae5b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.468477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb4ac1-deca-4bf1-bfc0-0080a6b7ae5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.468740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeb4ac1-deca-4bf1-bfc0-0080a6b7ae5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.469389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb4ac1-deca-4bf1-bfc0-0080a6b7ae5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.471009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeb4ac1-deca-4bf1-bfc0-0080a6b7ae5b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.483358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 975/1953] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:17:52.487539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e59e4-24b7-496a-94a3-d287977894d8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.497486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e59e4-24b7-496a-94a3-d287977894d8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.502587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5429d-8d4f-435e-9de7-9941cf636823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.503228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5429d-8d4f-435e-9de7-9941cf636823 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.503573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5429d-8d4f-435e-9de7-9941cf636823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.503843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5429d-8d4f-435e-9de7-9941cf636823 admin admin] PATH_INFO: `/v3/projects/cbb47d61872f45d0b3bd5d04eb233b05/users/46a27bed033b45ea951dd83801b1c3e7/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.519104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5429d-8d4f-435e-9de7-9941cf636823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb47d61872f45d0b3bd5d04eb233b05, user_id=46a27bed033b45ea951dd83801b1c3e7, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.521818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5429d-8d4f-435e-9de7-9941cf636823 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.536998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 979/1954] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:52 2022] PUT /identity/v3/projects/cbb47d61872f45d0b3bd5d04eb233b05/users/46a27bed033b45ea951dd83801b1c3e7/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:52.547638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e723f-0f13-4605-9fe1-c5ed13f46ffc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.547918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e723f-0f13-4605-9fe1-c5ed13f46ffc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.548186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107e723f-0f13-4605-9fe1-c5ed13f46ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.576792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-107e723f-0f13-4605-9fe1-c5ed13f46ffc None None] MFA Rules not processed for user `46a27bed033b45ea951dd83801b1c3e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:52.578100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-107e723f-0f13-4605-9fe1-c5ed13f46ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:52.656976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 976/1955] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:52 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:52.661649 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5429d-8d4f-435e-9de7-9941cf636823 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.744729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5429d-8d4f-435e-9de7-9941cf636823 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.750626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b1e9-9c7d-45e7-9c05-c67854df8df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.751239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b1e9-9c7d-45e7-9c05-c67854df8df0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.751489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b1e9-9c7d-45e7-9c05-c67854df8df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.751754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552b1e9-9c7d-45e7-9c05-c67854df8df0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.752384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552b1e9-9c7d-45e7-9c05-c67854df8df0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.754350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552b1e9-9c7d-45e7-9c05-c67854df8df0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.786609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 980/1956] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:17:52 2022] POST /identity/v3/projects => generated 409 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:52.787788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e723f-0f13-4605-9fe1-c5ed13f46ffc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.798638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107e723f-0f13-4605-9fe1-c5ed13f46ffc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.859445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:52.870341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a8294-a163-4f7a-aa2f-6600deefe508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.871083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a8294-a163-4f7a-aa2f-6600deefe508 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.871425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a8294-a163-4f7a-aa2f-6600deefe508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.871693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a8294-a163-4f7a-aa2f-6600deefe508 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.872356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a8294-a163-4f7a-aa2f-6600deefe508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.874316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a8294-a163-4f7a-aa2f-6600deefe508 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.902190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 977/1957] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:17:52 2022] POST /identity/v3/users => generated 311 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:17:52.903405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b1e9-9c7d-45e7-9c05-c67854df8df0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.915390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552b1e9-9c7d-45e7-9c05-c67854df8df0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.920866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55087f-dba8-4b2a-8376-ac882bb9807d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.921566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55087f-dba8-4b2a-8376-ac882bb9807d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.921960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55087f-dba8-4b2a-8376-ac882bb9807d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.922293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf55087f-dba8-4b2a-8376-ac882bb9807d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.923027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55087f-dba8-4b2a-8376-ac882bb9807d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.924751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf55087f-dba8-4b2a-8376-ac882bb9807d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.935849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 981/1958] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:52.940167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a8294-a163-4f7a-aa2f-6600deefe508 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:52.956213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a8294-a163-4f7a-aa2f-6600deefe508 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:52.961311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff212ee-b693-4188-af85-f8df7fe03408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:52.962078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff212ee-b693-4188-af85-f8df7fe03408 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:52.962351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff212ee-b693-4188-af85-f8df7fe03408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:52.962582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff212ee-b693-4188-af85-f8df7fe03408 admin admin] PATH_INFO: `/v3/projects/0b2adafbd6364531a112a6d2566a879a/users/f2df3bcf50e44c28ab33a58385374dab/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:52.978454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff212ee-b693-4188-af85-f8df7fe03408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2adafbd6364531a112a6d2566a879a, user_id=f2df3bcf50e44c28ab33a58385374dab, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:52.981084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff212ee-b693-4188-af85-f8df7fe03408 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:52.996060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 978/1959] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:52 2022] PUT /identity/v3/projects/0b2adafbd6364531a112a6d2566a879a/users/f2df3bcf50e44c28ab33a58385374dab/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:53.000180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55087f-dba8-4b2a-8376-ac882bb9807d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:53.011320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf55087f-dba8-4b2a-8376-ac882bb9807d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:53.069503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42dc05d-6db6-4307-b7dd-6ab9032c9dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:53.070148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dc05d-6db6-4307-b7dd-6ab9032c9dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:53.070422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dc05d-6db6-4307-b7dd-6ab9032c9dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:53.070698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42dc05d-6db6-4307-b7dd-6ab9032c9dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:53.071400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42dc05d-6db6-4307-b7dd-6ab9032c9dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:53.072995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42dc05d-6db6-4307-b7dd-6ab9032c9dc1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:53.082508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 982/1960] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:17:52 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:17:53.088937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff212ee-b693-4188-af85-f8df7fe03408 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:53.102771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff212ee-b693-4188-af85-f8df7fe03408 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:53.108207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:53.108862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:53.109132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:53.109397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] PATH_INFO: `/v3/projects/0b2adafbd6364531a112a6d2566a879a/users/f2df3bcf50e44c28ab33a58385374dab/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:53.111511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b2adafbd6364531a112a6d2566a879a, user_id=f2df3bcf50e44c28ab33a58385374dab, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:53.114179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:53.125278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 979/1961] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:17:53 2022] PUT /identity/v3/projects/0b2adafbd6364531a112a6d2566a879a/users/f2df3bcf50e44c28ab33a58385374dab/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:17:53.136505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452cc1c-819b-45db-b30e-155abf60c73a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:53.136925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452cc1c-819b-45db-b30e-155abf60c73a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:53.137309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5452cc1c-819b-45db-b30e-155abf60c73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:53.166132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5452cc1c-819b-45db-b30e-155abf60c73a None None] MFA Rules not processed for user `f2df3bcf50e44c28ab33a58385374dab`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:17:53.167361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5452cc1c-819b-45db-b30e-155abf60c73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:53.176642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:17:53.246659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 983/1962] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:17:53 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:17:53.257875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:53.259943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:17:53.323040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761dac1-450a-4f6b-89c4-a4e0fcb03c6d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:53.326916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:53.327641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:53.328042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:53.328182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:53.341837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:53.343265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:53.353596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 980/1963] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:53 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:17:53.534714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452cc1c-819b-45db-b30e-155abf60c73a None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:17:53.545070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5452cc1c-819b-45db-b30e-155abf60c73a None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:17:53.549803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058258cc-7519-487e-99fa-f7300797f529 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:17:53.550415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058258cc-7519-487e-99fa-f7300797f529 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:17:53.550691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058258cc-7519-487e-99fa-f7300797f529 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:17:53.550958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058258cc-7519-487e-99fa-f7300797f529 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:17:53.611288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058258cc-7519-487e-99fa-f7300797f529 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:17:53.612775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058258cc-7519-487e-99fa-f7300797f529 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:17:53.622922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 984/1964] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:17:53 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:18:03.349087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=45,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:18:03.618241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=53 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:18:19.871529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:18:19.873094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:18:19.935002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b52257-41ec-477d-827f-dac1df3d2dfc service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:18:19.938842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322106d1-0681-4136-bffa-d8abd6901c54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:18:19.939455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322106d1-0681-4136-bffa-d8abd6901c54 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:18:19.939725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322106d1-0681-4136-bffa-d8abd6901c54 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:18:19.939995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322106d1-0681-4136-bffa-d8abd6901c54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:18:19.941736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-322106d1-0681-4136-bffa-d8abd6901c54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:18:19.951583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322106d1-0681-4136-bffa-d8abd6901c54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:18:19.953039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322106d1-0681-4136-bffa-d8abd6901c54 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:18:19.961176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 981/1965] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:18:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:18:29.957915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=26 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:19.451052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db59516-e399-410e-bf0e-2ad346a525fa None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:19.451509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db59516-e399-410e-bf0e-2ad346a525fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:19.451620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db59516-e399-410e-bf0e-2ad346a525fa None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:19.452994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 985/1966] 10.222.0.23 () {54 vars in 961 bytes} [Wed Oct 5 14:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:20:19.458212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4422a-a568-4728-8e4d-fbb35e3ce097 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:19.458653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4422a-a568-4728-8e4d-fbb35e3ce097 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:19.458743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4422a-a568-4728-8e4d-fbb35e3ce097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:19.541880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-70f4422a-a568-4728-8e4d-fbb35e3ce097 None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:20:19.543115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-70f4422a-a568-4728-8e4d-fbb35e3ce097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:19.661410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 982/1967] 10.222.0.23 () {58 vars in 1058 bytes} [Wed Oct 5 14:20:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:20:19.669356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db59516-e399-410e-bf0e-2ad346a525fa None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:19.684923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db59516-e399-410e-bf0e-2ad346a525fa None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:19.690248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:19.690921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:19.691185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:19.691454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:19.693199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:19.710848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:19.712650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:19.725744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 986/1968] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:20:21.567013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4422a-a568-4728-8e4d-fbb35e3ce097 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:21.586881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4422a-a568-4728-8e4d-fbb35e3ce097 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:21.590408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:21.591012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:21.591284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:21.591546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:21.593345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:21.640798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:21.694088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:21.695753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:21.716944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 983/1969] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:20:22.995294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:22.996740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:23.023024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb770e90-a6f5-4701-9151-e2d42bf00ede service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:23.027307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abafec8-4648-49d5-ba91-ea74a90a39ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:23.028008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abafec8-4648-49d5-ba91-ea74a90a39ca service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:23.028343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abafec8-4648-49d5-ba91-ea74a90a39ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:23.028781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abafec8-4648-49d5-ba91-ea74a90a39ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:23.030653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0abafec8-4648-49d5-ba91-ea74a90a39ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:23.045163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abafec8-4648-49d5-ba91-ea74a90a39ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:23.046762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abafec8-4648-49d5-ba91-ea74a90a39ca service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:23.058641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 987/1970] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:20:23.382892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:23.384845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:23.414386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e17ac53-67ae-44c4-9ac3-483ec5f8796a service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:23.424536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:23.425470 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:23.425850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:23.426199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:23.428064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:23.443935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:23.446094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:23.459015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 984/1971] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:20:27.661412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721cbc3-7c4b-489a-adbd-3c52ff1e66a3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:27.662037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721cbc3-7c4b-489a-adbd-3c52ff1e66a3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:27.662037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721cbc3-7c4b-489a-adbd-3c52ff1e66a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:27.682432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f721cbc3-7c4b-489a-adbd-3c52ff1e66a3 None None] MFA Rules not processed for user `2e988e0435464c1d8aa4842fab56dfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:20:27.683658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f721cbc3-7c4b-489a-adbd-3c52ff1e66a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:27.738313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 988/1972] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:20:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:20:28.165323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:28.175675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ea257-091d-40a0-84f0-77be8be35e60 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:28.180203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f65af1-54d2-4824-b396-066cbc58d7f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:28.180832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f65af1-54d2-4824-b396-066cbc58d7f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:28.181106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f65af1-54d2-4824-b396-066cbc58d7f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:28.181370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f65af1-54d2-4824-b396-066cbc58d7f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:28.183262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-24f65af1-54d2-4824-b396-066cbc58d7f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:28.194050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f65af1-54d2-4824-b396-066cbc58d7f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:28.195506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f65af1-54d2-4824-b396-066cbc58d7f0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:28.206979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 985/1973] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:20:28.537377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721cbc3-7c4b-489a-adbd-3c52ff1e66a3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:28.550974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721cbc3-7c4b-489a-adbd-3c52ff1e66a3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:28.556831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:28.557459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:28.557767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:28.558037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:28.559856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:28.573588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:28.575383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:28.586813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 989/1974] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:20:31.048396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8c852-ca6c-479a-98b8-7061024c5b10 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:31.049568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8c852-ca6c-479a-98b8-7061024c5b10 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:31.050074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8c852-ca6c-479a-98b8-7061024c5b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:31.069864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-70d8c852-ca6c-479a-98b8-7061024c5b10 None None] MFA Rules not processed for user `bad6116fbd5c407c9f44084fadd9531e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:20:31.071167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-70d8c852-ca6c-479a-98b8-7061024c5b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:31.141248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 986/1975] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:20:31.479227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:31.488755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc67dd16-e1da-4014-81ef-4c65e2a89fb4 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:31.495868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:31.496541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:31.496819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:31.497091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:31.498963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:31.509701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:31.511286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:31.519527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 990/1976] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:20:36.254716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8c852-ca6c-479a-98b8-7061024c5b10 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:36.256482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-70d8c852-ca6c-479a-98b8-7061024c5b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:36.273336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8c852-ca6c-479a-98b8-7061024c5b10 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:36.280004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:36.280670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:36.281015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:36.281387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:36.283775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:36.299656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:36.301379 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:36.316891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 987/1977] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:20:36.638253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:20:36.654285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd072db4-e6a1-48c1-86c7-d72f7f94b5c1 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:20:36.660618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:20:36.661241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:20:36.661514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:20:36.661807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:20:36.663807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:20:36.676380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:20:36.678279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:20:36.690396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 991/1978] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:20:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:20:46.309893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=67 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:20:46.684967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=60 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:21:26.076819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:21:26.078438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:21:26.117325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6ecc2-4d35-4fc2-a867-32b09e30a2ec service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:21:26.121466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:21:26.122108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:21:26.122381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:21:26.122646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:21:26.124773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:21:26.210250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:21:26.211732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:21:26.232524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 988/1979] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:21:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:21:26.703497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:21:26.715748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e969a7-b9ce-41d2-a172-b87ee5ef0a72 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:21:26.721897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:21:26.722644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:21:26.722991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:21:26.723323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:21:26.725984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:21:26.817973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:21:26.820076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:21:26.839526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 992/1980] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:21:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:21:29.177489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:21:29.192267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a87039-f23d-40a2-98a4-3ac9d059c7ed service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:21:29.199020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:21:29.199598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:21:29.199929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:21:29.200186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:21:29.203433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:21:29.292764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:21:29.294137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:21:29.312847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 989/1981] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:21:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:21:36.837089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=41 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:21:39.310350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:21:58.390010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:21:58.402612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac4531-91b3-44c3-a900-44404cc74839 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:21:58.408162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:21:58.408802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:21:58.409076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:21:58.409344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:21:58.411100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:21:58.444013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:21:58.445562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:21:58.463968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 993/1982] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:21:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:22:01.384373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:22:01.398432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f892c0-4ff6-4011-acfe-2b4abde6fb55 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:22:01.404987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:22:01.405591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:22:01.405899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:22:01.406165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:22:01.408094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:22:01.442278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:22:01.443887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:22:01.464665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 990/1983] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:22:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:22:08.461394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=19 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:22:11.461433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=19 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:22:37.605868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:22:37.615607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1b760-d3b1-48a9-accb-42a33c947174 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:22:37.620193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:22:37.620789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:22:37.621110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:22:37.621316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:22:37.623306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:22:37.683601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:22:37.685037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:22:37.693907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 994/1984] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:22:47.690104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=28 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:22:49.317411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:22:49.333317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240b3f3-89a7-46d6-8ce9-27b4a7adf8f5 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:22:49.339922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:22:49.340501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:22:49.340765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:22:49.341189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:22:49.343530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:22:49.405870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:22:49.407325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:22:49.416148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 991/1985] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:22:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:22:53.572717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:22:53.581877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5a380-c905-4961-aebf-20c538ae19f1 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:22:53.586205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f91ed-72e5-4835-a015-8fa2232c1ace service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:22:53.586818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f91ed-72e5-4835-a015-8fa2232c1ace service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:22:53.587088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f91ed-72e5-4835-a015-8fa2232c1ace service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:22:53.587350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f91ed-72e5-4835-a015-8fa2232c1ace service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:22:53.602519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f91ed-72e5-4835-a015-8fa2232c1ace service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:22:53.603975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f91ed-72e5-4835-a015-8fa2232c1ace service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:22:53.613378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 995/1986] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:22:53 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:22:59.412361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=28 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:23:03.609200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:23:24.097332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:24.108076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2ed9e8-dc82-4812-8c39-19532b6b05f8 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:24.112955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c9aa1-3a5b-496c-9496-1e3011890349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:24.113578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c9aa1-3a5b-496c-9496-1e3011890349 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:24.113875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c9aa1-3a5b-496c-9496-1e3011890349 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:24.114145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c9aa1-3a5b-496c-9496-1e3011890349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:24.121623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c9aa1-3a5b-496c-9496-1e3011890349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:24.123078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c9aa1-3a5b-496c-9496-1e3011890349 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:24.132164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 992/1987] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:23:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:23:34.127577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:23:43.587138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f91ed-72e5-4835-a015-8fa2232c1ace service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:43.597926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f91ed-72e5-4835-a015-8fa2232c1ace service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:43.602768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:43.603402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:43.603658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:43.603922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:43.605705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:43.704568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:43.706214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:43.725821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 996/1988] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:23:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:23:44.446582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c9aa1-3a5b-496c-9496-1e3011890349 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:44.456672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c9aa1-3a5b-496c-9496-1e3011890349 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:44.461370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:44.462008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:44.462284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:44.462549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:44.464718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:44.556605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:44.558096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:44.577004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 993/1989] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:23:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:23:53.109284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:53.118181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c6209-ac35-4161-8c28-d9468b3702d6 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:53.123220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6335b63c-f4b3-465c-82b4-9034b4d0f54b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:53.124555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335b63c-f4b3-465c-82b4-9034b4d0f54b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:53.124902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335b63c-f4b3-465c-82b4-9034b4d0f54b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:53.125242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6335b63c-f4b3-465c-82b4-9034b4d0f54b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:53.139584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6335b63c-f4b3-465c-82b4-9034b4d0f54b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:53.141029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6335b63c-f4b3-465c-82b4-9034b4d0f54b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:53.151226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 997/1990] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:23:53 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:23:53.316857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:53.408502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1457a0-cd38-4e80-8dea-7d3434adcc52 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:53.413396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:53.414132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:53.414799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:53.415139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] PATH_INFO: `/v3/users/46a27bed033b45ea951dd83801b1c3e7` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:53.416560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=46a27bed033b45ea951dd83801b1c3e7)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:53.418610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:53.467932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46a27bed033b45ea951dd83801b1c3e7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:53.470025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:23:53.476220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46a27bed033b45ea951dd83801b1c3e7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:53.477138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46a27bed033b45ea951dd83801b1c3e7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:53.486850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46a27bed033b45ea951dd83801b1c3e7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:53.501219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46a27bed033b45ea951dd83801b1c3e7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:53.512854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 994/1991] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:23:53 2022] DELETE /identity/v3/users/46a27bed033b45ea951dd83801b1c3e7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:23:53.517567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701f478-c3bf-40af-955e-fed0eff62c88 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:53.517878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701f478-c3bf-40af-955e-fed0eff62c88 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:53.518148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701f478-c3bf-40af-955e-fed0eff62c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:53.558569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4701f478-c3bf-40af-955e-fed0eff62c88 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:23:53.559794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4701f478-c3bf-40af-955e-fed0eff62c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:53.622955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 998/1992] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:23:53.633709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:53.635220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:53.700184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22ef84-b288-45d7-b57e-2b1b2b729bca admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:53.704351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17aaa96-ba95-4f1c-a95d-e0da2d96c662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:53.704955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17aaa96-ba95-4f1c-a95d-e0da2d96c662 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:53.705224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17aaa96-ba95-4f1c-a95d-e0da2d96c662 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:53.705489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17aaa96-ba95-4f1c-a95d-e0da2d96c662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:53.714980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17aaa96-ba95-4f1c-a95d-e0da2d96c662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:53.716404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17aaa96-ba95-4f1c-a95d-e0da2d96c662 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:53.726610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 995/1993] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:23:53.998619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701f478-c3bf-40af-955e-fed0eff62c88 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:54.000197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4701f478-c3bf-40af-955e-fed0eff62c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:54.012374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701f478-c3bf-40af-955e-fed0eff62c88 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:54.018444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:54.019082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:54.019358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:54.019724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] PATH_INFO: `/v3/projects/cbb47d61872f45d0b3bd5d04eb233b05` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:54.020730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb47d61872f45d0b3bd5d04eb233b05)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:54.024015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:54.061941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:23:54.075411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb47d61872f45d0b3bd5d04eb233b05', 'request_id': 'req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.080676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb47d61872f45d0b3bd5d04eb233b05', 'request_id': 'req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.086769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb47d61872f45d0b3bd5d04eb233b05', 'request_id': 'req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.101972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] The token cache is being invalidate because project cbb47d61872f45d0b3bd5d04eb233b05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:23:54.103689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.104337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.105886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 999/1994] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:23:53 2022] DELETE /identity/v3/projects/cbb47d61872f45d0b3bd5d04eb233b05 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:23:54.109976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17aaa96-ba95-4f1c-a95d-e0da2d96c662 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:54.119844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17aaa96-ba95-4f1c-a95d-e0da2d96c662 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:54.183762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:54.184401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:54.184671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:54.184935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] PATH_INFO: `/v3/users/f2df3bcf50e44c28ab33a58385374dab` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:54.186087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2df3bcf50e44c28ab33a58385374dab)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:54.187994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:54.236068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2df3bcf50e44c28ab33a58385374dab'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.243109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2df3bcf50e44c28ab33a58385374dab'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.243769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2df3bcf50e44c28ab33a58385374dab'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.252430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2df3bcf50e44c28ab33a58385374dab'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.262059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:23:54.268850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3ac3d085-e144-4ecd-b24a-f879178ba083 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2df3bcf50e44c28ab33a58385374dab'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.275668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 996/1995] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:23:54 2022] DELETE /identity/v3/users/f2df3bcf50e44c28ab33a58385374dab => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:23:54.558522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:54.560082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:54.580332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6158a28a-dbbf-4ba1-a548-0ef636ce6a17 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:54.636925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:54.637548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:54.637839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:54.638115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] PATH_INFO: `/v3/projects/0b2adafbd6364531a112a6d2566a879a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:54.639222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b2adafbd6364531a112a6d2566a879a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:54.641131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:54.689363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b2adafbd6364531a112a6d2566a879a', 'request_id': 'req-4e9b08ad-07f8-4d71-9b7e-62967274755a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.694809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b2adafbd6364531a112a6d2566a879a', 'request_id': 'req-4e9b08ad-07f8-4d71-9b7e-62967274755a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.700886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b2adafbd6364531a112a6d2566a879a', 'request_id': 'req-4e9b08ad-07f8-4d71-9b7e-62967274755a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.714303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] The token cache is being invalidate because project 0b2adafbd6364531a112a6d2566a879a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:23:54.714587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.715231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4e9b08ad-07f8-4d71-9b7e-62967274755a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:23:54.716734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1000/1996] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:23:54 2022] DELETE /identity/v3/projects/0b2adafbd6364531a112a6d2566a879a => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:23:54.723998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c393af9-5fe7-47ae-8b61-daacde4f4b39 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:54.724503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c393af9-5fe7-47ae-8b61-daacde4f4b39 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:54.724610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c393af9-5fe7-47ae-8b61-daacde4f4b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:54.762876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3c393af9-5fe7-47ae-8b61-daacde4f4b39 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:23:54.764078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3c393af9-5fe7-47ae-8b61-daacde4f4b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:54.824995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 997/1997] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:23:54.891687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d846b-9cbb-463a-9702-2f206bdac3a4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:54.891912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d846b-9cbb-463a-9702-2f206bdac3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:54.892125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d846b-9cbb-463a-9702-2f206bdac3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:54.914672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d81d846b-9cbb-463a-9702-2f206bdac3a4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:23:54.915885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d81d846b-9cbb-463a-9702-2f206bdac3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:54.925251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1001/1998] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:23:54.929668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c393af9-5fe7-47ae-8b61-daacde4f4b39 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:54.939645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c393af9-5fe7-47ae-8b61-daacde4f4b39 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:54.945663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:54.946292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:54.946569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:54.946861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:54.947509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:23:54.947805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:54.949391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:54.961719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 998/1999] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:23:54.966199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d846b-9cbb-463a-9702-2f206bdac3a4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:54.976221 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d846b-9cbb-463a-9702-2f206bdac3a4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:54.982057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0295796-a9f2-4f8b-b429-4a89f018d2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:54.982654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0295796-a9f2-4f8b-b429-4a89f018d2bb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:54.982900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0295796-a9f2-4f8b-b429-4a89f018d2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:54.983196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0295796-a9f2-4f8b-b429-4a89f018d2bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:54.983819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0295796-a9f2-4f8b-b429-4a89f018d2bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:54.985751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0295796-a9f2-4f8b-b429-4a89f018d2bb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:55.026084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1002/2000] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:23:54 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:23:55.030647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:55.041080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b54aab5-1b15-4bee-8ab9-0ff1fc2f5621 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:55.098775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8529d-54fb-4210-a1a5-0959ee252718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:55.099384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8529d-54fb-4210-a1a5-0959ee252718 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:55.099710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8529d-54fb-4210-a1a5-0959ee252718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:55.099978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e8529d-54fb-4210-a1a5-0959ee252718 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:55.100607 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8529d-54fb-4210-a1a5-0959ee252718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:55.102557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e8529d-54fb-4210-a1a5-0959ee252718 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:55.126714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 999/2001] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:23:55 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:23:55.131038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0295796-a9f2-4f8b-b429-4a89f018d2bb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:55.141264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0295796-a9f2-4f8b-b429-4a89f018d2bb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:55.146200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133070d-6ef1-419e-8568-0d837bc15db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:55.146769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133070d-6ef1-419e-8568-0d837bc15db7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:55.147041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133070d-6ef1-419e-8568-0d837bc15db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:55.147313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4133070d-6ef1-419e-8568-0d837bc15db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:55.147948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133070d-6ef1-419e-8568-0d837bc15db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:55.149541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4133070d-6ef1-419e-8568-0d837bc15db7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:55.154803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:23:55.159664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1003/2002] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:23:55.163731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8529d-54fb-4210-a1a5-0959ee252718 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:55.174551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e8529d-54fb-4210-a1a5-0959ee252718 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:55.180366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69914860-3d96-4b52-b7dd-590709962246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:55.181057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69914860-3d96-4b52-b7dd-590709962246 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:55.181393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69914860-3d96-4b52-b7dd-590709962246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:55.181776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69914860-3d96-4b52-b7dd-590709962246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:55.182524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69914860-3d96-4b52-b7dd-590709962246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:55.184171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69914860-3d96-4b52-b7dd-590709962246 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:55.196198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1000/2003] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:23:55.200244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133070d-6ef1-419e-8568-0d837bc15db7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:55.210460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4133070d-6ef1-419e-8568-0d837bc15db7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:55.215973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:55.216743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:55.217084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:55.217410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] PATH_INFO: `/v3/projects/c31f6dd24d6545deb494f3cde27a9361/users/afa431bc26d84e42b93c0f127df7e39a/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:55.233954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c31f6dd24d6545deb494f3cde27a9361, user_id=afa431bc26d84e42b93c0f127df7e39a, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:55.236822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:55.253381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1004/2004] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:23:55 2022] PUT /identity/v3/projects/c31f6dd24d6545deb494f3cde27a9361/users/afa431bc26d84e42b93c0f127df7e39a/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:23:55.263542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb101e-e966-41a2-ab9e-c49a0a612ffe None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:55.263826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb101e-e966-41a2-ab9e-c49a0a612ffe None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:55.264090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb101e-e966-41a2-ab9e-c49a0a612ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:55.291868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-23bb101e-e966-41a2-ab9e-c49a0a612ffe None None] MFA Rules not processed for user `afa431bc26d84e42b93c0f127df7e39a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:23:55.293297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-23bb101e-e966-41a2-ab9e-c49a0a612ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:55.325324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:23:55.365173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1001/2005] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:23:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:23:55.376352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:23:55.378293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:23:55.440076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e090c29-99c3-4aa7-b52e-86ba45dde30f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:23:55.444946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49f903b-b0df-44f0-9ee6-115fc8263d02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:23:55.445866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f903b-b0df-44f0-9ee6-115fc8263d02 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:23:55.446139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f903b-b0df-44f0-9ee6-115fc8263d02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:23:55.446403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49f903b-b0df-44f0-9ee6-115fc8263d02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:23:55.455652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f903b-b0df-44f0-9ee6-115fc8263d02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:23:55.457085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49f903b-b0df-44f0-9ee6-115fc8263d02 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:23:55.466690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1005/2006] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:23:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:24:05.362324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=14 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:24:05.462273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:24:56.879140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb101e-e966-41a2-ab9e-c49a0a612ffe None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:24:56.880719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-23bb101e-e966-41a2-ab9e-c49a0a612ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:24:56.939532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb101e-e966-41a2-ab9e-c49a0a612ffe None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:24:56.944869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c668694-7c9c-4850-b943-16bc31ef7252 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:24:56.945486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c668694-7c9c-4850-b943-16bc31ef7252 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:24:56.945797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c668694-7c9c-4850-b943-16bc31ef7252 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:24:56.946063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c668694-7c9c-4850-b943-16bc31ef7252 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:24:56.948018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9c668694-7c9c-4850-b943-16bc31ef7252 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:24:57.011380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c668694-7c9c-4850-b943-16bc31ef7252 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:24:57.012883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c668694-7c9c-4850-b943-16bc31ef7252 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:24:57.022677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1002/2007] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:24:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:25:00.762411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b348-611a-47f0-bc82-14b5461c565b None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:00.763005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b348-611a-47f0-bc82-14b5461c565b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:00.763005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b348-611a-47f0-bc82-14b5461c565b None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:00.764706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1006/2008] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 14:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:25:00.794866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b015dda-9fe6-4412-8019-b9f09d7a282b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:00.795355 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b015dda-9fe6-4412-8019-b9f09d7a282b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:00.795481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b015dda-9fe6-4412-8019-b9f09d7a282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:00.817628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0b015dda-9fe6-4412-8019-b9f09d7a282b None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:25:00.819082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b015dda-9fe6-4412-8019-b9f09d7a282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:00.832982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b348-611a-47f0-bc82-14b5461c565b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:00.842942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a755b348-611a-47f0-bc82-14b5461c565b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:00.870782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b348-611a-47f0-bc82-14b5461c565b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:00.876214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:00.876979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:00.877255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:00.877525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] PATH_INFO: `/v3/users/596b52fe5595438d8acd593c61c91f67` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:00.878790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=596b52fe5595438d8acd593c61c91f67)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:00.880770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:00.893155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1003/2009] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 14:25:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:25:00.906718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b015dda-9fe6-4412-8019-b9f09d7a282b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:00.908468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0b015dda-9fe6-4412-8019-b9f09d7a282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:00.923089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b015dda-9fe6-4412-8019-b9f09d7a282b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:00.928413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a81f73-bb6d-41e6-9b5e-7347b90f1f19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:00.929045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a81f73-bb6d-41e6-9b5e-7347b90f1f19 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:00.929321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a81f73-bb6d-41e6-9b5e-7347b90f1f19 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:00.929594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a81f73-bb6d-41e6-9b5e-7347b90f1f19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:00.931613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596b52fe5595438d8acd593c61c91f67'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:00.938052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a81f73-bb6d-41e6-9b5e-7347b90f1f19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:00.938762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596b52fe5595438d8acd593c61c91f67'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:00.939710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a81f73-bb6d-41e6-9b5e-7347b90f1f19 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:00.940061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596b52fe5595438d8acd593c61c91f67'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:00.948580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596b52fe5595438d8acd593c61c91f67'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:00.949661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1004/2010] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:25:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:25:00.964244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7ed7bf4a-0a2d-4600-8f5f-8aaa7ed1ed59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596b52fe5595438d8acd593c61c91f67'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:00.970690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1007/2011] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:25:00 2022] DELETE /identity/v3/users/596b52fe5595438d8acd593c61c91f67 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:25:01.147549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a81f73-bb6d-41e6-9b5e-7347b90f1f19 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:01.158311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a81f73-bb6d-41e6-9b5e-7347b90f1f19 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:01.191874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:01.218518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:01.219151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.219423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.219689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] PATH_INFO: `/v3/projects/04395d53d99840c3b16d49e2f6d62631` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.221008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04395d53d99840c3b16d49e2f6d62631)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:01.223164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:01.285268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04395d53d99840c3b16d49e2f6d62631', 'request_id': 'req-ce3b8807-e12a-49b3-bfe9-51c518709aa9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:01.291413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04395d53d99840c3b16d49e2f6d62631', 'request_id': 'req-ce3b8807-e12a-49b3-bfe9-51c518709aa9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:01.297853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04395d53d99840c3b16d49e2f6d62631', 'request_id': 'req-ce3b8807-e12a-49b3-bfe9-51c518709aa9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:01.322059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] The token cache is being invalidate because project 04395d53d99840c3b16d49e2f6d62631 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:25:01.322059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:01.322059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ce3b8807-e12a-49b3-bfe9-51c518709aa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:25:01.322059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1005/2012] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:25:01 2022] DELETE /identity/v3/projects/04395d53d99840c3b16d49e2f6d62631 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:25:01.322439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001db2c-0e44-469a-895f-4062918e7f67 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.322439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001db2c-0e44-469a-895f-4062918e7f67 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.322439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8001db2c-0e44-469a-895f-4062918e7f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.341412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8001db2c-0e44-469a-895f-4062918e7f67 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:25:01.342570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8001db2c-0e44-469a-895f-4062918e7f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:01.401378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1008/2013] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:25:01.411246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa096cbd-8769-400c-b61a-7f73796f9d48 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.411531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa096cbd-8769-400c-b61a-7f73796f9d48 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.411805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa096cbd-8769-400c-b61a-7f73796f9d48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.433230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-aa096cbd-8769-400c-b61a-7f73796f9d48 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:25:01.434483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aa096cbd-8769-400c-b61a-7f73796f9d48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:01.443891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1006/2014] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:25:01.448433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001db2c-0e44-469a-895f-4062918e7f67 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:01.459738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8001db2c-0e44-469a-895f-4062918e7f67 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:01.468218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:01.468218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.468218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.468218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.468748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:25:01.469042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:01.470814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:01.494334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1009/2015] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:25:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:25:01.494593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa096cbd-8769-400c-b61a-7f73796f9d48 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:01.500311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa096cbd-8769-400c-b61a-7f73796f9d48 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:01.506871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aefb6-62e0-481b-9d46-14a31ddf25a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:01.507513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aefb6-62e0-481b-9d46-14a31ddf25a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.507789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aefb6-62e0-481b-9d46-14a31ddf25a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.508056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aefb6-62e0-481b-9d46-14a31ddf25a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.508957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aefb6-62e0-481b-9d46-14a31ddf25a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:01.511032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aefb6-62e0-481b-9d46-14a31ddf25a6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:01.550608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1007/2016] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:25:01 2022] POST /identity/v3/projects => generated 391 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:25:01.556146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:01.566183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa468ee-1dbe-4d66-98b1-ff3cdbf232e4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:01.624774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d8a86-f362-430a-94cb-9f4e682e4f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:01.625383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d8a86-f362-430a-94cb-9f4e682e4f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.625656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d8a86-f362-430a-94cb-9f4e682e4f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.625944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d8a86-f362-430a-94cb-9f4e682e4f39 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.626772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d8a86-f362-430a-94cb-9f4e682e4f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:01.628705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389d8a86-f362-430a-94cb-9f4e682e4f39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:01.636731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:01.652540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1010/2017] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:25:01 2022] POST /identity/v3/users => generated 296 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:25:01.656937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aefb6-62e0-481b-9d46-14a31ddf25a6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:01.667336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aefb6-62e0-481b-9d46-14a31ddf25a6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:01.673209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afe772-bbda-4319-ae5f-e6946740a3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:01.673939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe772-bbda-4319-ae5f-e6946740a3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.674226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe772-bbda-4319-ae5f-e6946740a3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.674490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afe772-bbda-4319-ae5f-e6946740a3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.675137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afe772-bbda-4319-ae5f-e6946740a3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:01.676737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afe772-bbda-4319-ae5f-e6946740a3c3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:01.687508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1008/2018] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:25:01.691641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d8a86-f362-430a-94cb-9f4e682e4f39 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:01.701892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d8a86-f362-430a-94cb-9f4e682e4f39 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:01.706889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fa7ab3-e6a3-44ca-861c-642c63ab1519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:01.707491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7ab3-e6a3-44ca-861c-642c63ab1519 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.707757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7ab3-e6a3-44ca-861c-642c63ab1519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.708024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fa7ab3-e6a3-44ca-861c-642c63ab1519 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.708685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa7ab3-e6a3-44ca-861c-642c63ab1519 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:01.710301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fa7ab3-e6a3-44ca-861c-642c63ab1519 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:01.721342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1011/2019] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:25:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:25:01.725954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afe772-bbda-4319-ae5f-e6946740a3c3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:01.736596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afe772-bbda-4319-ae5f-e6946740a3c3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:01.742324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:01.742983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.743260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.743530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] PATH_INFO: `/v3/projects/ae3d0e9a5de8481fa19a58f7780a4b7a/users/6e4a13a6981f4e0898dc22ea8525a1ee/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.758443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3d0e9a5de8481fa19a58f7780a4b7a, user_id=6e4a13a6981f4e0898dc22ea8525a1ee, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:01.761556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:01.776795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1009/2020] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:25:01 2022] PUT /identity/v3/projects/ae3d0e9a5de8481fa19a58f7780a4b7a/users/6e4a13a6981f4e0898dc22ea8525a1ee/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:25:01.782234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05daae-0705-43d1-ac81-5921b633fd96 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.782822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05daae-0705-43d1-ac81-5921b633fd96 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.782928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b05daae-0705-43d1-ac81-5921b633fd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.801508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7b05daae-0705-43d1-ac81-5921b633fd96 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:25:01.802727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7b05daae-0705-43d1-ac81-5921b633fd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:01.864694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1012/2021] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:25:01.874558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:01.876540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:01.939413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b2bd2-ab2d-4d48-8bbd-c10bd7c6df67 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:01.943557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad494c4a-d49e-4ad2-aada-91a0f9f32b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:01.944179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad494c4a-d49e-4ad2-aada-91a0f9f32b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:01.944450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad494c4a-d49e-4ad2-aada-91a0f9f32b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:01.944721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad494c4a-d49e-4ad2-aada-91a0f9f32b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:01.957807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad494c4a-d49e-4ad2-aada-91a0f9f32b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:01.959257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad494c4a-d49e-4ad2-aada-91a0f9f32b5a service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:01.969212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1010/2022] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:25:05.804039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf042da7-7493-4d3c-bf35-3300e6815cca None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:05.804571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf042da7-7493-4d3c-bf35-3300e6815cca None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:05.804571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf042da7-7493-4d3c-bf35-3300e6815cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:05.835183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-bf042da7-7493-4d3c-bf35-3300e6815cca None None] MFA Rules not processed for user `6e4a13a6981f4e0898dc22ea8525a1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:25:05.836404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf042da7-7493-4d3c-bf35-3300e6815cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:05.911710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1013/2023] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:25:05.921983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3cb46-481e-4f1d-924c-7e51ce87ed2c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:05.922509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3cb46-481e-4f1d-924c-7e51ce87ed2c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:05.922509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff3cb46-481e-4f1d-924c-7e51ce87ed2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:05.930270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:05.944559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cff3cb46-481e-4f1d-924c-7e51ce87ed2c None None] MFA Rules not processed for user `6e4a13a6981f4e0898dc22ea8525a1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:25:05.946462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cff3cb46-481e-4f1d-924c-7e51ce87ed2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:05.953918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1011/2024] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:25:05.968007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf042da7-7493-4d3c-bf35-3300e6815cca None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:05.969549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bf042da7-7493-4d3c-bf35-3300e6815cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:06.045392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf042da7-7493-4d3c-bf35-3300e6815cca None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:06.051920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:06.052529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:06.052811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:06.053067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:06.057930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:06.062045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:06.064002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:06.075464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1014/2025] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:25:05 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:25:06.797049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111aa11-4556-4aca-8779-3a51f341745f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:06.797332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111aa11-4556-4aca-8779-3a51f341745f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:06.797600 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1111aa11-4556-4aca-8779-3a51f341745f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:06.823908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1111aa11-4556-4aca-8779-3a51f341745f None None] MFA Rules not processed for user `6e4a13a6981f4e0898dc22ea8525a1ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:25:06.825752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1111aa11-4556-4aca-8779-3a51f341745f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:06.838753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1012/2026] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:25:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:25:16.072723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:16.816946 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:20.906866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:20.908664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:20.920913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eff8e5-b19f-4dd3-84ab-842eb73a4bfc service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:20.924727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:20.925362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:20.925649 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:20.925927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:20.927650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:20.940736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:20.942350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:20.953008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1015/2027] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:25:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:25:24.941678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111aa11-4556-4aca-8779-3a51f341745f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:24.952111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1111aa11-4556-4aca-8779-3a51f341745f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:24.956586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:24.957245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:24.957520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:24.957858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:24.959875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:24.981951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:24.983529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:24.995306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1013/2028] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:25:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:25:26.386912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:25:26.388490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:26.460910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef5286d-52b0-4818-9b6a-1892eda5da0b service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:25:26.466588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:25:26.467210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:25:26.467491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:25:26.467769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:25:26.469703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:25:26.482528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:25:26.484114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:25:26.494522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1016/2029] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:25:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:25:34.990665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:25:36.491094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=38 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:26:27.129730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:26:27.131365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:26:27.190795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9da5356-b960-4031-8707-7ed95aec558c service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:26:27.195384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:26:27.196011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:26:27.196285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:26:27.196553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:26:27.198657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:26:27.272859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:26:27.274697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:26:27.284703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1014/2030] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:26:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:26:29.770636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:26:29.772294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:26:29.785144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261fbca2-b145-4dc9-a18b-5fa177d19453 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:26:29.789181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:26:29.789821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:26:29.790093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:26:29.790483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:26:29.792118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:26:29.851433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:26:29.852884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:26:29.861943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1017/2031] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:26:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:26:37.280530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=48 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:26:39.857996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=28 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:26:58.373527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:26:58.383262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ed13a4-50f6-4baf-9622-cb1a45096865 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:26:58.387105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:26:58.387714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:26:58.387986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:26:58.388252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:26:58.390084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:26:58.456093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:26:58.457537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:26:58.466446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1015/2032] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:26:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:27:01.297949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:27:01.312278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468cf0f-17c1-484b-938f-4330e2b6bf09 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:27:01.318334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:27:01.319151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:27:01.319437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:27:01.319699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:27:01.321772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:27:01.394620 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:27:01.396050 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:27:01.405306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1018/2033] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:27:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:27:08.462519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=28 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:27:11.401176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=28 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:27:36.774456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:27:36.783802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e2019-b653-4b44-b17b-70186074ebbc service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:27:36.787611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:27:36.788338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:27:36.788687 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:27:36.789021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:27:36.790958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:27:36.871052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:27:36.872551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:27:36.890791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1016/2034] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:27:36 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:27:46.888423 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=41 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:27:49.393150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:27:49.407539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa49e07-42ec-44dd-87cc-bb818aecded1 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:27:49.413872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c3b33a-0fd5-4836-961c-fc58795297b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:27:49.414513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3b33a-0fd5-4836-961c-fc58795297b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:27:49.414747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3b33a-0fd5-4836-961c-fc58795297b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:27:49.415035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c3b33a-0fd5-4836-961c-fc58795297b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:27:49.417107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-23c3b33a-0fd5-4836-961c-fc58795297b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:27:49.499486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3b33a-0fd5-4836-961c-fc58795297b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:27:49.501030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c3b33a-0fd5-4836-961c-fc58795297b1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:27:49.519915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1019/2035] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:27:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:27:59.517091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=41 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:28:01.276445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:28:01.278443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:01.296538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174457a9-9469-4560-8b5c-4018a9cbe72e service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:28:01.303240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:28:01.303985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:28:01.304319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:28:01.304782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:28:01.307587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:01.321793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:28:01.323398 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:28:01.335251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1017/2036] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:28:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:28:04.745245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff38d0e-a5cc-453d-9e19-4d0b227fec76 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:28:04.749474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff38d0e-a5cc-453d-9e19-4d0b227fec76 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:28:04.749794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff38d0e-a5cc-453d-9e19-4d0b227fec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:28:04.778425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ff38d0e-a5cc-453d-9e19-4d0b227fec76 None None] MFA Rules not processed for user `2e988e0435464c1d8aa4842fab56dfa4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:28:04.780026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ff38d0e-a5cc-453d-9e19-4d0b227fec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:04.788271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1020/2037] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:28:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:28:05.115475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:28:05.126314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b5d59-41a3-4253-a65f-dcd6d3cc863e service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:28:05.129951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:28:05.130605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:28:05.130878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:28:05.131141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:28:05.132949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:05.143206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:28:05.144671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:28:05.152839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1018/2038] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:28:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:28:06.490882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff38d0e-a5cc-453d-9e19-4d0b227fec76 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:28:06.492678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ff38d0e-a5cc-453d-9e19-4d0b227fec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:06.510100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff38d0e-a5cc-453d-9e19-4d0b227fec76 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:28:06.518326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:28:06.519037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:28:06.519441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:28:06.519795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:28:06.521627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:06.538701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:28:06.540298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:28:06.557429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1021/2039] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:28:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:28:15.149269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:28:16.547554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=12 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:28:18.898705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:28:18.917739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39542e8b-e1d2-4d64-9a08-0e4a1ea80c88 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:28:18.924476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:28:18.925102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:28:18.925377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:28:18.925773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:28:18.927438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:18.943815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:28:18.945529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:28:18.965495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1019/2040] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:28:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:28:23.147861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:28:23.160540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b671d-4e72-42fd-b0b1-b6c6e6ba330b service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:28:23.171437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:28:23.172163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:28:23.172515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:28:23.172850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:28:23.175053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:23.188312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:28:23.189976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:28:23.202366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1022/2041] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:28:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:28:23.695196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:28:23.704638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d580fca9-9f7e-4fc6-bd21-2840fa49aa38 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:28:23.708604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:28:23.709206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:28:23.709478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:28:23.709755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:28:23.713507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:23.726761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:28:23.728230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:28:23.736666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1020/2042] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:28:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:28:33.197766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:28:33.733047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:28:55.152920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:28:55.162781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f56085-e1b8-4eb3-b14d-77e7d3b63d19 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:28:55.167222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65512a1-78e1-481f-9255-f99efb14c66f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:28:55.167823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65512a1-78e1-481f-9255-f99efb14c66f service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:28:55.168093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65512a1-78e1-481f-9255-f99efb14c66f service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:28:55.168354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65512a1-78e1-481f-9255-f99efb14c66f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:28:55.170368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b65512a1-78e1-481f-9255-f99efb14c66f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:28:55.232622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65512a1-78e1-481f-9255-f99efb14c66f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:28:55.234100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65512a1-78e1-481f-9255-f99efb14c66f service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:28:55.243429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1023/2043] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:28:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:05.239193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=28 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:39.002539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:39.004606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:39.022435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a2fa86-25ab-4153-b826-2792654dfa36 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:39.027386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:39.028025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:39.028307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:39.028574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] PATH_INFO: `/v3/users/083c32f884054b7897702ac2735df8cd` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:39.029796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=083c32f884054b7897702ac2735df8cd)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:39.031702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:39.088358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '083c32f884054b7897702ac2735df8cd'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.095122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '083c32f884054b7897702ac2735df8cd'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.095777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '083c32f884054b7897702ac2735df8cd'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.104944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '083c32f884054b7897702ac2735df8cd'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.116766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '083c32f884054b7897702ac2735df8cd'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.122775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1021/2044] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:29:39 2022] DELETE /identity/v3/users/083c32f884054b7897702ac2735df8cd => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:39.127338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3ece6-de29-4faf-834c-fe8022210a04 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:39.127839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3ece6-de29-4faf-834c-fe8022210a04 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:39.127942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3ece6-de29-4faf-834c-fe8022210a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:39.152671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4aa3ece6-de29-4faf-834c-fe8022210a04 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:39.153934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4aa3ece6-de29-4faf-834c-fe8022210a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:39.215125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1024/2045] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:39.225160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:39.226705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:39.308975 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542bdc79-f50a-4147-bfd9-2007e8f97e10 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:39.315445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190dcce-f738-4bad-a3f3-6b25f9f41349 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:39.316064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190dcce-f738-4bad-a3f3-6b25f9f41349 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:39.316360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190dcce-f738-4bad-a3f3-6b25f9f41349 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:39.316646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6190dcce-f738-4bad-a3f3-6b25f9f41349 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:39.327376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190dcce-f738-4bad-a3f3-6b25f9f41349 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:39.328866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6190dcce-f738-4bad-a3f3-6b25f9f41349 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:39.338826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1022/2046] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:39.610512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3ece6-de29-4faf-834c-fe8022210a04 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:39.612104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4aa3ece6-de29-4faf-834c-fe8022210a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:39.624670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3ece6-de29-4faf-834c-fe8022210a04 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:39.643532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:39.643532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:39.643532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:39.643532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] PATH_INFO: `/v3/projects/d46197f77cf946b690cc5c4bf69d45e9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:39.643989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d46197f77cf946b690cc5c4bf69d45e9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:39.643989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:39.701216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd46197f77cf946b690cc5c4bf69d45e9', 'request_id': 'req-a2940d4f-f9e4-4b10-ba13-859465ac202d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.709823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd46197f77cf946b690cc5c4bf69d45e9', 'request_id': 'req-a2940d4f-f9e4-4b10-ba13-859465ac202d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.716778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd46197f77cf946b690cc5c4bf69d45e9', 'request_id': 'req-a2940d4f-f9e4-4b10-ba13-859465ac202d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.731571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] The token cache is being invalidate because project d46197f77cf946b690cc5c4bf69d45e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:29:39.731870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.732494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.734079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1025/2047] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:29:39 2022] DELETE /identity/v3/projects/d46197f77cf946b690cc5c4bf69d45e9 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:39.738502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190dcce-f738-4bad-a3f3-6b25f9f41349 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:39.749267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6190dcce-f738-4bad-a3f3-6b25f9f41349 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:39.821042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:39.821726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:39.822003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:39.822271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] PATH_INFO: `/v3/users/1e76b3b679b94888a7f4f7bb956be821` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:39.836714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e76b3b679b94888a7f4f7bb956be821)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:39.838743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:39.843136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:39.888944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e76b3b679b94888a7f4f7bb956be821'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.895917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e76b3b679b94888a7f4f7bb956be821'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.896578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e76b3b679b94888a7f4f7bb956be821'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.906433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e76b3b679b94888a7f4f7bb956be821'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.919288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e76b3b679b94888a7f4f7bb956be821'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:39.925742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1023/2048] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:29:39 2022] DELETE /identity/v3/users/1e76b3b679b94888a7f4f7bb956be821 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:40.218607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:40.220062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:40.234427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2940d4f-f9e4-4b10-ba13-859465ac202d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:40.310927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:40.311584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:40.312031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:40.312487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] PATH_INFO: `/v3/projects/e7156d160e8944209bf025d5adca1ef2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:40.326252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7156d160e8944209bf025d5adca1ef2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:40.328192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:40.345793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:40.377045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7156d160e8944209bf025d5adca1ef2', 'request_id': 'req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.382914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7156d160e8944209bf025d5adca1ef2', 'request_id': 'req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.390173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7156d160e8944209bf025d5adca1ef2', 'request_id': 'req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.404109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] The token cache is being invalidate because project e7156d160e8944209bf025d5adca1ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:29:40.404396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.405098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.406774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1026/2049] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:29:40 2022] DELETE /identity/v3/projects/e7156d160e8944209bf025d5adca1ef2 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:40.410873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:40.412299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:40.425117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29138d7-82bf-4406-a23a-b0462dd614ea admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:40.488253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:40.488962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:40.489222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:40.489493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] PATH_INFO: `/v3/users/dd2fc33f04c848aba970da169d356b31` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:40.503350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2fc33f04c848aba970da169d356b31)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:40.505485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:40.557271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2fc33f04c848aba970da169d356b31'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.564916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2fc33f04c848aba970da169d356b31'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.565602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2fc33f04c848aba970da169d356b31'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.575067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2fc33f04c848aba970da169d356b31'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.583271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:40.588374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fdcf85eb-5e0b-44f8-8719-63a83e5200c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2fc33f04c848aba970da169d356b31'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:40.595633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1024/2050] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:29:40 2022] DELETE /identity/v3/users/dd2fc33f04c848aba970da169d356b31 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:40.894924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:40.896957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:40.917973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfaf6b-2ca7-458a-9805-06ff0ae3bb31 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:40.986530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:40.987184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:40.987485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:40.987753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] PATH_INFO: `/v3/projects/12b40bf4d234457aae486730758d50ec` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:40.999317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b40bf4d234457aae486730758d50ec)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:41.001262 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:41.050356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b40bf4d234457aae486730758d50ec', 'request_id': 'req-1b579583-a176-4117-b6b8-e4355ba92abc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:41.055835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b40bf4d234457aae486730758d50ec', 'request_id': 'req-1b579583-a176-4117-b6b8-e4355ba92abc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:41.062431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b40bf4d234457aae486730758d50ec', 'request_id': 'req-1b579583-a176-4117-b6b8-e4355ba92abc'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:41.076725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] The token cache is being invalidate because project 12b40bf4d234457aae486730758d50ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:29:41.077011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:41.077621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1b579583-a176-4117-b6b8-e4355ba92abc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:41.079164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1027/2051] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:29:40 2022] DELETE /identity/v3/projects/12b40bf4d234457aae486730758d50ec => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:41.085035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b0e9-cd1c-4ceb-8875-7e7be405f59a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.085319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b0e9-cd1c-4ceb-8875-7e7be405f59a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.085623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b0e9-cd1c-4ceb-8875-7e7be405f59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.107391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-34c0b0e9-cd1c-4ceb-8875-7e7be405f59a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:41.108541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34c0b0e9-cd1c-4ceb-8875-7e7be405f59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:41.170071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1025/2052] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:29:41.183343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642b1a-47c0-4ebc-8882-f1897f163fed None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.183635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642b1a-47c0-4ebc-8882-f1897f163fed None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.183917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35642b1a-47c0-4ebc-8882-f1897f163fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.208471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-35642b1a-47c0-4ebc-8882-f1897f163fed None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:41.209781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-35642b1a-47c0-4ebc-8882-f1897f163fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:41.218451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1028/2053] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:41.222657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b0e9-cd1c-4ceb-8875-7e7be405f59a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:41.233047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b0e9-cd1c-4ceb-8875-7e7be405f59a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:41.237620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:41.238259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.238530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.238792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.239437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:29:41.239740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:41.241350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:41.252812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1026/2054] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:29:41.257120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35642b1a-47c0-4ebc-8882-f1897f163fed None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:41.267134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35642b1a-47c0-4ebc-8882-f1897f163fed None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:41.272131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff74d32-f75d-4f50-83e9-5c7f483bbae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:41.272737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff74d32-f75d-4f50-83e9-5c7f483bbae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.273006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff74d32-f75d-4f50-83e9-5c7f483bbae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.273315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff74d32-f75d-4f50-83e9-5c7f483bbae4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.274000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff74d32-f75d-4f50-83e9-5c7f483bbae4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:41.275896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff74d32-f75d-4f50-83e9-5c7f483bbae4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:41.299758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:41.308098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1029/2055] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:29:41 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:29:41.313478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:41.323546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192fd91c-88a6-4993-8e9b-04dea28580f4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:41.383985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d263f7-f3f9-4a3f-9f50-b5db1e5dbcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:41.384559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d263f7-f3f9-4a3f-9f50-b5db1e5dbcce admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.384787 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d263f7-f3f9-4a3f-9f50-b5db1e5dbcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.385038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d263f7-f3f9-4a3f-9f50-b5db1e5dbcce admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.385713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d263f7-f3f9-4a3f-9f50-b5db1e5dbcce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:41.387756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d263f7-f3f9-4a3f-9f50-b5db1e5dbcce admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:41.413838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1027/2056] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:29:41 2022] POST /identity/v3/users => generated 286 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:29:41.424495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff74d32-f75d-4f50-83e9-5c7f483bbae4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:41.437591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff74d32-f75d-4f50-83e9-5c7f483bbae4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:41.443428 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4579bc-2315-4a04-ad4c-f246d2f55228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:41.444030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4579bc-2315-4a04-ad4c-f246d2f55228 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.444299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4579bc-2315-4a04-ad4c-f246d2f55228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.444557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4579bc-2315-4a04-ad4c-f246d2f55228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.445256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4579bc-2315-4a04-ad4c-f246d2f55228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:41.446908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4579bc-2315-4a04-ad4c-f246d2f55228 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:41.461589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1030/2057] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:29:41.468349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d263f7-f3f9-4a3f-9f50-b5db1e5dbcce admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:41.480070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d263f7-f3f9-4a3f-9f50-b5db1e5dbcce admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:41.485812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665d7a7a-b98d-4160-b71e-947b1d465fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:41.486425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7a7a-b98d-4160-b71e-947b1d465fb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.486695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7a7a-b98d-4160-b71e-947b1d465fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.486959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665d7a7a-b98d-4160-b71e-947b1d465fb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.487598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d7a7a-b98d-4160-b71e-947b1d465fb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:41.489269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665d7a7a-b98d-4160-b71e-947b1d465fb7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:41.501418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1028/2058] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:29:41 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:29:41.505889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4579bc-2315-4a04-ad4c-f246d2f55228 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:41.520312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4579bc-2315-4a04-ad4c-f246d2f55228 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:41.525676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:41.526358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.526637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.526908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] PATH_INFO: `/v3/projects/edc99ad3772145ce808197eaf98cb5a3/users/c6b2448558194fbfb02e3e76bad02b06/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.542583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc99ad3772145ce808197eaf98cb5a3, user_id=c6b2448558194fbfb02e3e76bad02b06, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:41.545564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:41.563436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1031/2059] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:29:41 2022] PUT /identity/v3/projects/edc99ad3772145ce808197eaf98cb5a3/users/c6b2448558194fbfb02e3e76bad02b06/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:41.574580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f39a-97fe-45c1-96b3-dcdc313db1dc None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.575231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f39a-97fe-45c1-96b3-dcdc313db1dc None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.575605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2f39a-97fe-45c1-96b3-dcdc313db1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.614376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b4f2f39a-97fe-45c1-96b3-dcdc313db1dc None None] MFA Rules not processed for user `c6b2448558194fbfb02e3e76bad02b06`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:41.616405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b4f2f39a-97fe-45c1-96b3-dcdc313db1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:41.668442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:41.691069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1029/2060] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:41.704858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:41.706804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:41.792669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e832996-b69b-47bf-91c0-4ab7abc20e9f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:41.796680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc66a2-8c2a-4122-8820-f8d6ee6c71f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417b661f770846f189c36d0804b6dce5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:41.797292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc66a2-8c2a-4122-8820-f8d6ee6c71f1 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:41.797563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc66a2-8c2a-4122-8820-f8d6ee6c71f1 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:41.797863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc66a2-8c2a-4122-8820-f8d6ee6c71f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:41.807001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc66a2-8c2a-4122-8820-f8d6ee6c71f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:41.808655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bc66a2-8c2a-4122-8820-f8d6ee6c71f1 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:41.817781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1032/2061] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:44.056731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2f39a-97fe-45c1-96b3-dcdc313db1dc None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:44.058501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b4f2f39a-97fe-45c1-96b3-dcdc313db1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:44.118959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2f39a-97fe-45c1-96b3-dcdc313db1dc None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:44.122865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:44.123478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:44.123757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:44.124022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:44.125916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:44.193119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:44.194627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:44.204430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1030/2062] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:50.458987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc66a2-8c2a-4122-8820-f8d6ee6c71f1 service glance] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:50.468759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc66a2-8c2a-4122-8820-f8d6ee6c71f1 service glance] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:50.472277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0b39ef-6886-47ea-acdb-1ba6dd4a3c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:50.472893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b39ef-6886-47ea-acdb-1ba6dd4a3c81 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:50.473175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b39ef-6886-47ea-acdb-1ba6dd4a3c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:50.473797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0b39ef-6886-47ea-acdb-1ba6dd4a3c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:50.475564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9d0b39ef-6886-47ea-acdb-1ba6dd4a3c81 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:50.576909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b39ef-6886-47ea-acdb-1ba6dd4a3c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:50.578527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0b39ef-6886-47ea-acdb-1ba6dd4a3c81 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:50.600403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1033/2063] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:51.068429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:51.070069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:51.138323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b479f4-70c3-4ab1-a50a-cd8542ba9b27 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:51.142710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:51.143348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:51.143628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:51.143903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] PATH_INFO: `/v3/users/b2f0f0ad39614af1bcf4bb97f44394ba` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:51.145016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2f0f0ad39614af1bcf4bb97f44394ba)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:51.146997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:51.195336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:51.203145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2f0f0ad39614af1bcf4bb97f44394ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.210829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2f0f0ad39614af1bcf4bb97f44394ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.211531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2f0f0ad39614af1bcf4bb97f44394ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.221918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2f0f0ad39614af1bcf4bb97f44394ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.239241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2f0f0ad39614af1bcf4bb97f44394ba'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.248683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1031/2064] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:29:51 2022] DELETE /identity/v3/users/b2f0f0ad39614af1bcf4bb97f44394ba => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:51.254015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ad39-4698-4f34-b5b7-cb16a4a7d802 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:51.254631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ad39-4698-4f34-b5b7-cb16a4a7d802 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:51.255138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8ad39-4698-4f34-b5b7-cb16a4a7d802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:51.277451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f0b8ad39-4698-4f34-b5b7-cb16a4a7d802 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:51.278729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f0b8ad39-4698-4f34-b5b7-cb16a4a7d802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:51.284119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:51.338691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1034/2065] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:51.349046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:51.351038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:51.410829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca7a105-6681-4c1a-88b9-b93ab2a36caf admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:51.414675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63388e93-51d6-4ed1-82fe-f91d8ae5feda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:51.415301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63388e93-51d6-4ed1-82fe-f91d8ae5feda service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:51.415614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63388e93-51d6-4ed1-82fe-f91d8ae5feda service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:51.415890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63388e93-51d6-4ed1-82fe-f91d8ae5feda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:51.425687 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63388e93-51d6-4ed1-82fe-f91d8ae5feda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:51.427148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63388e93-51d6-4ed1-82fe-f91d8ae5feda service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:51.436940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1032/2066] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:51.699773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8ad39-4698-4f34-b5b7-cb16a4a7d802 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:51.702068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f0b8ad39-4698-4f34-b5b7-cb16a4a7d802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:51.714996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8ad39-4698-4f34-b5b7-cb16a4a7d802 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:51.720099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:51.720688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:51.721116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:51.721394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] PATH_INFO: `/v3/projects/e0bf8d8baf034324bd9cdfae771a1fb3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:51.722549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0bf8d8baf034324bd9cdfae771a1fb3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:51.724821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:51.774417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0bf8d8baf034324bd9cdfae771a1fb3', 'request_id': 'req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.779933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0bf8d8baf034324bd9cdfae771a1fb3', 'request_id': 'req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.786489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0bf8d8baf034324bd9cdfae771a1fb3', 'request_id': 'req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.800025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] The token cache is being invalidate because project e0bf8d8baf034324bd9cdfae771a1fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:29:51.800311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.800949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.802727 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1035/2067] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:29:51 2022] DELETE /identity/v3/projects/e0bf8d8baf034324bd9cdfae771a1fb3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:51.806572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63388e93-51d6-4ed1-82fe-f91d8ae5feda service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:51.816981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63388e93-51d6-4ed1-82fe-f91d8ae5feda service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:51.884870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:51.885505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:51.885810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:51.886086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] PATH_INFO: `/v3/users/481ac6f1439240c18fdfc968b5bd2240` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:51.900524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=481ac6f1439240c18fdfc968b5bd2240)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:51.902890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:51.951614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '481ac6f1439240c18fdfc968b5bd2240'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.954090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:51.960911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '481ac6f1439240c18fdfc968b5bd2240'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.961621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '481ac6f1439240c18fdfc968b5bd2240'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.970162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '481ac6f1439240c18fdfc968b5bd2240'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.982619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '481ac6f1439240c18fdfc968b5bd2240'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:51.988927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1033/2068] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:29:51 2022] DELETE /identity/v3/users/481ac6f1439240c18fdfc968b5bd2240 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:29:52.262018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:52.263542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:52.277609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fabae-a96b-4a0e-ad84-2e24dae8ed10 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:52.341151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:52.341881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:52.342157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:52.342545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] PATH_INFO: `/v3/projects/d300498e17aa45dcbd164221dbec880c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:52.354188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d300498e17aa45dcbd164221dbec880c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:52.356146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:52.392636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:52.412980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd300498e17aa45dcbd164221dbec880c', 'request_id': 'req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.418763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd300498e17aa45dcbd164221dbec880c', 'request_id': 'req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.425364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd300498e17aa45dcbd164221dbec880c', 'request_id': 'req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.439174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] The token cache is being invalidate because project d300498e17aa45dcbd164221dbec880c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:29:52.439459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.440072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.441782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1036/2069] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:29:52 2022] DELETE /identity/v3/projects/d300498e17aa45dcbd164221dbec880c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:52.445564 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:52.447104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:52.460771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1278ddb4-02c5-4c4a-928c-5187130e0b1f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:52.534977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:52.535616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:52.535894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:52.536164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] PATH_INFO: `/v3/users/d4985d664d6441f8893e12140e19329f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:52.537321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4985d664d6441f8893e12140e19329f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:52.539266 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:52.586689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4985d664d6441f8893e12140e19329f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.594115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4985d664d6441f8893e12140e19329f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.594968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4985d664d6441f8893e12140e19329f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.603801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4985d664d6441f8893e12140e19329f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.615214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9f8801e4-6b91-4e40-ab60-50932ad59ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4985d664d6441f8893e12140e19329f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:52.620988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1034/2070] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:29:52 2022] DELETE /identity/v3/users/d4985d664d6441f8893e12140e19329f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:52.904208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:52.905778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:52.921218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58c9e2-4966-4cf2-9d81-49c9ad10c2a4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:52.989437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:52.990019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:52.990293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:52.990549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] PATH_INFO: `/v3/projects/00215c4d262949c0a4e9f81647504104` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:52.991674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=00215c4d262949c0a4e9f81647504104)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:52.993575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.049216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00215c4d262949c0a4e9f81647504104', 'request_id': 'req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:53.055131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00215c4d262949c0a4e9f81647504104', 'request_id': 'req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:53.061253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00215c4d262949c0a4e9f81647504104', 'request_id': 'req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:53.075722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] The token cache is being invalidate because project 00215c4d262949c0a4e9f81647504104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:29:53.076002 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:53.076610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-696f4d21-3ed1-44f3-9b19-b8df71b2ccbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:53.078234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1037/2071] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:29:52 2022] DELETE /identity/v3/projects/00215c4d262949c0a4e9f81647504104 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:53.084575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2241d-6536-4393-8abc-a4caab7bdbf0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.084861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2241d-6536-4393-8abc-a4caab7bdbf0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.085135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2241d-6536-4393-8abc-a4caab7bdbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.105500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-51a2241d-6536-4393-8abc-a4caab7bdbf0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:53.106748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51a2241d-6536-4393-8abc-a4caab7bdbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:53.149480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:53.168660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1035/2072] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:53.177086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ed84-7cef-41cc-8672-c2949f323efc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.177365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ed84-7cef-41cc-8672-c2949f323efc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.177656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1ed84-7cef-41cc-8672-c2949f323efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.198417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9eb1ed84-7cef-41cc-8672-c2949f323efc None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:53.199673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9eb1ed84-7cef-41cc-8672-c2949f323efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:53.209026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1038/2073] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:53.213589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2241d-6536-4393-8abc-a4caab7bdbf0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.225269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2241d-6536-4393-8abc-a4caab7bdbf0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.230344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.230964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.231237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.231512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.232156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:29:53.232454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.234152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.246125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1036/2074] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:29:53.250523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1ed84-7cef-41cc-8672-c2949f323efc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.259846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb1ed84-7cef-41cc-8672-c2949f323efc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.264911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ec2bd-827f-4ce0-8122-54481b863da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.265601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ec2bd-827f-4ce0-8122-54481b863da9 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.265911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ec2bd-827f-4ce0-8122-54481b863da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.266176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ec2bd-827f-4ce0-8122-54481b863da9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.266798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ec2bd-827f-4ce0-8122-54481b863da9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.268704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832ec2bd-827f-4ce0-8122-54481b863da9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.299292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1039/2075] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:29:53 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:29:53.303440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.314864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8183a9-36f7-4ab9-8fa2-92ae928a58b7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.374714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6292c4-9f38-49c7-97b1-78d44e6535ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.375373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6292c4-9f38-49c7-97b1-78d44e6535ad admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.375657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6292c4-9f38-49c7-97b1-78d44e6535ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.375924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6292c4-9f38-49c7-97b1-78d44e6535ad admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.376694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6292c4-9f38-49c7-97b1-78d44e6535ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.378638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6292c4-9f38-49c7-97b1-78d44e6535ad admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.411451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1037/2076] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:29:53 2022] POST /identity/v3/users => generated 282 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:29:53.412886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ec2bd-827f-4ce0-8122-54481b863da9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.423813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832ec2bd-827f-4ce0-8122-54481b863da9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.428774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a132d7-57cb-4e31-a136-dcc6ee874ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.429375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a132d7-57cb-4e31-a136-dcc6ee874ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.429697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a132d7-57cb-4e31-a136-dcc6ee874ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.429985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a132d7-57cb-4e31-a136-dcc6ee874ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.430623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a132d7-57cb-4e31-a136-dcc6ee874ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.432227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a132d7-57cb-4e31-a136-dcc6ee874ad1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.443571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1040/2077] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:29:53.447879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6292c4-9f38-49c7-97b1-78d44e6535ad admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.458632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6292c4-9f38-49c7-97b1-78d44e6535ad admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.463521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db20ccd-a554-4d38-9438-f2a409704004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.464131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20ccd-a554-4d38-9438-f2a409704004 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.464403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20ccd-a554-4d38-9438-f2a409704004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.464666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db20ccd-a554-4d38-9438-f2a409704004 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.465321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db20ccd-a554-4d38-9438-f2a409704004 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.466933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db20ccd-a554-4d38-9438-f2a409704004 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.479291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1038/2078] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:29:53.483236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a132d7-57cb-4e31-a136-dcc6ee874ad1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.493387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a132d7-57cb-4e31-a136-dcc6ee874ad1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.498370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5dffb-27c5-4fe8-9c86-8959baaa7e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.498950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5dffb-27c5-4fe8-9c86-8959baaa7e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.499219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5dffb-27c5-4fe8-9c86-8959baaa7e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.499470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf5dffb-27c5-4fe8-9c86-8959baaa7e55 admin admin] PATH_INFO: `/v3/projects/db540aeb30d64a73856441da6a1e0431/users/fcc151de897b4297898677a651cb0723/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.502224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:53.516690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5dffb-27c5-4fe8-9c86-8959baaa7e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db540aeb30d64a73856441da6a1e0431, user_id=fcc151de897b4297898677a651cb0723, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.519566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf5dffb-27c5-4fe8-9c86-8959baaa7e55 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.534680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1041/2079] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:29:53 2022] PUT /identity/v3/projects/db540aeb30d64a73856441da6a1e0431/users/fcc151de897b4297898677a651cb0723/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:53.607063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f04cf-9a86-484f-ae13-7ae4d612a7c8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.607364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f04cf-9a86-484f-ae13-7ae4d612a7c8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.607730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4f04cf-9a86-484f-ae13-7ae4d612a7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.641984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4e4f04cf-9a86-484f-ae13-7ae4d612a7c8 None None] MFA Rules not processed for user `fcc151de897b4297898677a651cb0723`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:53.643193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4e4f04cf-9a86-484f-ae13-7ae4d612a7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:53.712826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1039/2080] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:53.717566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5dffb-27c5-4fe8-9c86-8959baaa7e55 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.781000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf5dffb-27c5-4fe8-9c86-8959baaa7e55 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.785926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4ecf-24f6-49c6-bd47-771609e76448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.786571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4ecf-24f6-49c6-bd47-771609e76448 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.786843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4ecf-24f6-49c6-bd47-771609e76448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.787112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d4ecf-24f6-49c6-bd47-771609e76448 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.787740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d4ecf-24f6-49c6-bd47-771609e76448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.789661 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d4ecf-24f6-49c6-bd47-771609e76448 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.818627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1042/2081] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:29:53 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:29:53.822794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f04cf-9a86-484f-ae13-7ae4d612a7c8 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.833590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4f04cf-9a86-484f-ae13-7ae4d612a7c8 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.874251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:53.893302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243a0c0-9fd5-41b5-b682-35b2fff2783e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.893930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243a0c0-9fd5-41b5-b682-35b2fff2783e admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.894206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243a0c0-9fd5-41b5-b682-35b2fff2783e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.894477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e243a0c0-9fd5-41b5-b682-35b2fff2783e admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.895227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243a0c0-9fd5-41b5-b682-35b2fff2783e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.897233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e243a0c0-9fd5-41b5-b682-35b2fff2783e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.919644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1040/2082] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:29:53 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:29:53.923872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4ecf-24f6-49c6-bd47-771609e76448 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.934733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d4ecf-24f6-49c6-bd47-771609e76448 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.939575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2a3b2-4af0-4523-9932-9c4913aee967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.940185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2a3b2-4af0-4523-9932-9c4913aee967 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.940493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2a3b2-4af0-4523-9932-9c4913aee967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.940736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f2a3b2-4af0-4523-9932-9c4913aee967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.941361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2a3b2-4af0-4523-9932-9c4913aee967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.943196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f2a3b2-4af0-4523-9932-9c4913aee967 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:53.954123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1043/2083] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:29:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:29:53.958643 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243a0c0-9fd5-41b5-b682-35b2fff2783e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:53.968930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e243a0c0-9fd5-41b5-b682-35b2fff2783e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:53.974454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8feab0-6edd-4057-bfaf-f2d49a2459aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:53.975133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feab0-6edd-4057-bfaf-f2d49a2459aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:53.975404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feab0-6edd-4057-bfaf-f2d49a2459aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:53.975671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8feab0-6edd-4057-bfaf-f2d49a2459aa admin admin] PATH_INFO: `/v3/projects/77f8523343b242018ffbc0e83569e093/users/ac92924b370b4e25afce97be0fd29f3f/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:53.990585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8feab0-6edd-4057-bfaf-f2d49a2459aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f8523343b242018ffbc0e83569e093, user_id=ac92924b370b4e25afce97be0fd29f3f, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:53.993217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8feab0-6edd-4057-bfaf-f2d49a2459aa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:54.012667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1041/2084] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:29:53 2022] PUT /identity/v3/projects/77f8523343b242018ffbc0e83569e093/users/ac92924b370b4e25afce97be0fd29f3f/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:54.017569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2a3b2-4af0-4523-9932-9c4913aee967 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:54.027729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f2a3b2-4af0-4523-9932-9c4913aee967 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:54.082393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57d5ee-95fb-4b88-ba57-e68d293e8384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:54.082992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57d5ee-95fb-4b88-ba57-e68d293e8384 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:54.083263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57d5ee-95fb-4b88-ba57-e68d293e8384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:54.083537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57d5ee-95fb-4b88-ba57-e68d293e8384 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:54.084153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57d5ee-95fb-4b88-ba57-e68d293e8384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:54.085787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57d5ee-95fb-4b88-ba57-e68d293e8384 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:54.095493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1044/2085] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:29:54.100081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8feab0-6edd-4057-bfaf-f2d49a2459aa admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:54.110937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8feab0-6edd-4057-bfaf-f2d49a2459aa admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:54.116165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:54.116819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:54.117090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:54.117358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] PATH_INFO: `/v3/projects/77f8523343b242018ffbc0e83569e093/users/ac92924b370b4e25afce97be0fd29f3f/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:54.119532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] RBAC: Authorizing `identity:create_grant(project_id=77f8523343b242018ffbc0e83569e093, user_id=ac92924b370b4e25afce97be0fd29f3f, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:54.122231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:54.133380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1042/2086] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:29:54 2022] PUT /identity/v3/projects/77f8523343b242018ffbc0e83569e093/users/ac92924b370b4e25afce97be0fd29f3f/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:54.143778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921cb1-bb82-4da4-9498-c7d611b4809f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:54.144113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921cb1-bb82-4da4-9498-c7d611b4809f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:54.144496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59921cb1-bb82-4da4-9498-c7d611b4809f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:54.178321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-59921cb1-bb82-4da4-9498-c7d611b4809f None None] MFA Rules not processed for user `ac92924b370b4e25afce97be0fd29f3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:54.179520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-59921cb1-bb82-4da4-9498-c7d611b4809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:54.223850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:54.254970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1045/2087] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:54.266680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:54.268699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:54.329308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4684f-c62f-49df-9deb-4e034192429b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:54.333432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:54.334069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:54.334337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:54.334602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:54.344079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:54.345545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:54.355293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1043/2088] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:54.453737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59921cb1-bb82-4da4-9498-c7d611b4809f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:54.453737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-59921cb1-bb82-4da4-9498-c7d611b4809f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:54.464778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59921cb1-bb82-4da4-9498-c7d611b4809f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:54.468731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee7e23-aba8-4541-82cf-1cc0c219ab0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:54.469350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7e23-aba8-4541-82cf-1cc0c219ab0f service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:54.469627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7e23-aba8-4541-82cf-1cc0c219ab0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:54.469922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee7e23-aba8-4541-82cf-1cc0c219ab0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:54.531627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee7e23-aba8-4541-82cf-1cc0c219ab0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:54.533092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee7e23-aba8-4541-82cf-1cc0c219ab0f service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:54.543878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1046/2089] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:55.746750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:55.748300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:55.825527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c85fee-109f-48d5-a90b-666373470017 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:55.831322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:55.831971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:55.832295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:55.832491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] PATH_INFO: `/v3/users/afa431bc26d84e42b93c0f127df7e39a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:55.833720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa431bc26d84e42b93c0f127df7e39a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:55.835639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:55.879015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:55.885407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa431bc26d84e42b93c0f127df7e39a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:55.893250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa431bc26d84e42b93c0f127df7e39a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:55.893934 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa431bc26d84e42b93c0f127df7e39a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:55.903209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa431bc26d84e42b93c0f127df7e39a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:55.915686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa431bc26d84e42b93c0f127df7e39a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:55.921627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1044/2090] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:29:55 2022] DELETE /identity/v3/users/afa431bc26d84e42b93c0f127df7e39a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:55.926470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970a29b-d396-47ed-ae52-5c926706bee3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:55.926953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970a29b-d396-47ed-ae52-5c926706bee3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:55.927065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970a29b-d396-47ed-ae52-5c926706bee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:55.945892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8970a29b-d396-47ed-ae52-5c926706bee3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:55.947039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8970a29b-d396-47ed-ae52-5c926706bee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:56.006385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1047/2091] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:56.016490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:56.018426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:56.077646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f203531f-a797-466f-aecc-decbb3aec2ad admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:56.081743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8bb12e-6e6d-48a2-b95a-1778977a17c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:56.082363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bb12e-6e6d-48a2-b95a-1778977a17c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.082636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bb12e-6e6d-48a2-b95a-1778977a17c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.082904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8bb12e-6e6d-48a2-b95a-1778977a17c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.092394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bb12e-6e6d-48a2-b95a-1778977a17c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:56.093939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8bb12e-6e6d-48a2-b95a-1778977a17c8 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:56.104137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1045/2092] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:29:56.387043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970a29b-d396-47ed-ae52-5c926706bee3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:56.388713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8970a29b-d396-47ed-ae52-5c926706bee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:56.405837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970a29b-d396-47ed-ae52-5c926706bee3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:56.411321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:56.411997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.412286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.412529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] PATH_INFO: `/v3/projects/c31f6dd24d6545deb494f3cde27a9361` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.413680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c31f6dd24d6545deb494f3cde27a9361)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:56.415769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:56.487181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c31f6dd24d6545deb494f3cde27a9361', 'request_id': 'req-bb379171-45dc-4d1a-87d4-4fc582b2dd40'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:56.493015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c31f6dd24d6545deb494f3cde27a9361', 'request_id': 'req-bb379171-45dc-4d1a-87d4-4fc582b2dd40'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:56.499434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c31f6dd24d6545deb494f3cde27a9361', 'request_id': 'req-bb379171-45dc-4d1a-87d4-4fc582b2dd40'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:56.510215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:56.514870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] The token cache is being invalidate because project c31f6dd24d6545deb494f3cde27a9361 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:29:56.515256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:56.516252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-bb379171-45dc-4d1a-87d4-4fc582b2dd40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:29:56.518190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1048/2093] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:29:56 2022] DELETE /identity/v3/projects/c31f6dd24d6545deb494f3cde27a9361 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:56.527159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9387a-e58f-40d1-8105-de8104e11e46 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.527446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9387a-e58f-40d1-8105-de8104e11e46 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.527717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d9387a-e58f-40d1-8105-de8104e11e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.548958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c1d9387a-e58f-40d1-8105-de8104e11e46 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:56.550187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c1d9387a-e58f-40d1-8105-de8104e11e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:56.609456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1046/2094] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:56.621872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2456c-576b-4775-b867-c6db311c1116 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.622165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2456c-576b-4775-b867-c6db311c1116 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.622483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2456c-576b-4775-b867-c6db311c1116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.647782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a0a2456c-576b-4775-b867-c6db311c1116 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:56.648993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a0a2456c-576b-4775-b867-c6db311c1116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:56.657451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1049/2095] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:29:56.661854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9387a-e58f-40d1-8105-de8104e11e46 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:56.672002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d9387a-e58f-40d1-8105-de8104e11e46 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:56.680807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:56.681440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.681753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.682036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.683028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:29:56.683344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:56.685026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:56.712067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1047/2096] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:29:56.713763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2456c-576b-4775-b867-c6db311c1116 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:56.723954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2456c-576b-4775-b867-c6db311c1116 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:56.729225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c86225-ca54-4276-910c-af773b15e1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:56.729878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c86225-ca54-4276-910c-af773b15e1bb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.730259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c86225-ca54-4276-910c-af773b15e1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.730534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c86225-ca54-4276-910c-af773b15e1bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.731223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c86225-ca54-4276-910c-af773b15e1bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:56.733141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c86225-ca54-4276-910c-af773b15e1bb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:56.767317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1050/2097] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:29:56 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:29:56.771638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:56.781709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e239cd-42c1-4c82-ad67-ad33e5c4087c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:56.826451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:29:56.842225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82509825-5af8-4f45-a605-381ac455b259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:56.842839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82509825-5af8-4f45-a605-381ac455b259 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.843118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82509825-5af8-4f45-a605-381ac455b259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.843375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82509825-5af8-4f45-a605-381ac455b259 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.844014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82509825-5af8-4f45-a605-381ac455b259 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:56.846148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82509825-5af8-4f45-a605-381ac455b259 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:56.871457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1048/2098] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:29:56 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:29:56.875809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c86225-ca54-4276-910c-af773b15e1bb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:56.887017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c86225-ca54-4276-910c-af773b15e1bb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:56.891832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2784b-e9b6-4102-aa60-70ceb4b85ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:56.892471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2784b-e9b6-4102-aa60-70ceb4b85ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.892740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2784b-e9b6-4102-aa60-70ceb4b85ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.892999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a2784b-e9b6-4102-aa60-70ceb4b85ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.893663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2784b-e9b6-4102-aa60-70ceb4b85ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:56.895382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a2784b-e9b6-4102-aa60-70ceb4b85ad8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:56.910296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1051/2099] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:29:56.911831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82509825-5af8-4f45-a605-381ac455b259 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:56.923432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82509825-5af8-4f45-a605-381ac455b259 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:56.927957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44b180e-b5b4-4e51-a16b-14237698a6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:56.928562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b180e-b5b4-4e51-a16b-14237698a6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.928836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b180e-b5b4-4e51-a16b-14237698a6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.929097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44b180e-b5b4-4e51-a16b-14237698a6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.929764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b180e-b5b4-4e51-a16b-14237698a6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:56.931360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44b180e-b5b4-4e51-a16b-14237698a6c4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:56.942201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1049/2100] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:29:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:29:56.946572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2784b-e9b6-4102-aa60-70ceb4b85ad8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:56.956791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a2784b-e9b6-4102-aa60-70ceb4b85ad8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:56.961503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:56.962195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:56.962469 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:56.962736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] PATH_INFO: `/v3/projects/126541d90e9c43269898b1249aa6ee9b/users/f109f3d7076543d6bf16eac46a40b3e2/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:56.978843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=126541d90e9c43269898b1249aa6ee9b, user_id=f109f3d7076543d6bf16eac46a40b3e2, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:56.981501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:56.997411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1052/2101] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:29:56 2022] PUT /identity/v3/projects/126541d90e9c43269898b1249aa6ee9b/users/f109f3d7076543d6bf16eac46a40b3e2/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:29:57.002516 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad39b2-d18f-4c8d-815a-fca2a441d2ba None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:57.002887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad39b2-d18f-4c8d-815a-fca2a441d2ba None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:57.003163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ad39b2-d18f-4c8d-815a-fca2a441d2ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:57.022275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-77ad39b2-d18f-4c8d-815a-fca2a441d2ba None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:29:57.023406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-77ad39b2-d18f-4c8d-815a-fca2a441d2ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:57.082294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1050/2102] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:29:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:29:57.092225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:29:57.094305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:29:57.159035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957f0a0-8dcb-41e1-90fe-f558a49a7aa7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:29:57.162909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e57bda-511c-477d-97b4-6877da70c6d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:29:57.163525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e57bda-511c-477d-97b4-6877da70c6d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:29:57.163804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e57bda-511c-477d-97b4-6877da70c6d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:29:57.164071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e57bda-511c-477d-97b4-6877da70c6d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:29:57.173679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e57bda-511c-477d-97b4-6877da70c6d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:29:57.175116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e57bda-511c-477d-97b4-6877da70c6d4 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:29:57.184894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1053/2103] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:30:01.090205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa367f0-6044-4a2c-b614-2f76147598cb None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:01.090780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa367f0-6044-4a2c-b614-2f76147598cb None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:01.090780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa367f0-6044-4a2c-b614-2f76147598cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:01.121657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4aa367f0-6044-4a2c-b614-2f76147598cb None None] MFA Rules not processed for user `f109f3d7076543d6bf16eac46a40b3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:30:01.122898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4aa367f0-6044-4a2c-b614-2f76147598cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:01.194825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1051/2104] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:30:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:30:01.196100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e57bda-511c-477d-97b4-6877da70c6d4 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:01.206761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e57bda-511c-477d-97b4-6877da70c6d4 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:01.212695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28554dc-37ed-4dfc-ae99-aed1f055cca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:01.213549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28554dc-37ed-4dfc-ae99-aed1f055cca1 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:01.214039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28554dc-37ed-4dfc-ae99-aed1f055cca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:01.214468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28554dc-37ed-4dfc-ae99-aed1f055cca1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:01.215568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28554dc-37ed-4dfc-ae99-aed1f055cca1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:01.217953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28554dc-37ed-4dfc-ae99-aed1f055cca1 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:01.254213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1054/2105] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:30:01 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:30:01.259503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa367f0-6044-4a2c-b614-2f76147598cb None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:01.270796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa367f0-6044-4a2c-b614-2f76147598cb None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:01.356823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:01.367816 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33fbbc-9a3f-4338-9306-3cfca7e9ab7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:01.368588 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33fbbc-9a3f-4338-9306-3cfca7e9ab7b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:01.368919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33fbbc-9a3f-4338-9306-3cfca7e9ab7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:01.369254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f33fbbc-9a3f-4338-9306-3cfca7e9ab7b admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:01.370570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33fbbc-9a3f-4338-9306-3cfca7e9ab7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:01.372231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f33fbbc-9a3f-4338-9306-3cfca7e9ab7b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:01.398549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1052/2106] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:30:01 2022] POST /identity/v3/users => generated 296 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:30:01.402856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28554dc-37ed-4dfc-ae99-aed1f055cca1 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:01.416347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28554dc-37ed-4dfc-ae99-aed1f055cca1 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:01.420927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21620b-a78c-4b2e-bc41-c871b70bf310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:01.421806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21620b-a78c-4b2e-bc41-c871b70bf310 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:01.422081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21620b-a78c-4b2e-bc41-c871b70bf310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:01.422533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21620b-a78c-4b2e-bc41-c871b70bf310 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:01.423202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21620b-a78c-4b2e-bc41-c871b70bf310 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:01.425266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21620b-a78c-4b2e-bc41-c871b70bf310 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:01.438319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1055/2107] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:30:01.442743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33fbbc-9a3f-4338-9306-3cfca7e9ab7b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:01.455353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f33fbbc-9a3f-4338-9306-3cfca7e9ab7b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:01.460373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce018cc-e067-4f5b-be30-e203a9d1b49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:01.461069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce018cc-e067-4f5b-be30-e203a9d1b49b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:01.461346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce018cc-e067-4f5b-be30-e203a9d1b49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:01.461611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce018cc-e067-4f5b-be30-e203a9d1b49b admin admin] PATH_INFO: `/v3/projects/b188c61d7d1242459d7ca94e7b3438b4/users/e438d2ca78f54d329bbd3dce41a9ff65/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:01.477680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce018cc-e067-4f5b-be30-e203a9d1b49b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b188c61d7d1242459d7ca94e7b3438b4, user_id=e438d2ca78f54d329bbd3dce41a9ff65, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:01.480449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce018cc-e067-4f5b-be30-e203a9d1b49b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:01.518258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1053/2108] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:30:01 2022] PUT /identity/v3/projects/b188c61d7d1242459d7ca94e7b3438b4/users/e438d2ca78f54d329bbd3dce41a9ff65/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:30:01.527220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21620b-a78c-4b2e-bc41-c871b70bf310 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:01.561538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21620b-a78c-4b2e-bc41-c871b70bf310 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:01.636812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:01.637522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:01.637877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:01.638143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:01.638786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:01.640388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:01.654642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1056/2109] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:30:01 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:30:01.659352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce018cc-e067-4f5b-be30-e203a9d1b49b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:01.672287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce018cc-e067-4f5b-be30-e203a9d1b49b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:01.677730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd23ae4-4d2f-402a-9612-80a120c06d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:01.678399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd23ae4-4d2f-402a-9612-80a120c06d45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:01.678679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd23ae4-4d2f-402a-9612-80a120c06d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:01.678949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd23ae4-4d2f-402a-9612-80a120c06d45 admin admin] PATH_INFO: `/v3/projects/b188c61d7d1242459d7ca94e7b3438b4/users/e438d2ca78f54d329bbd3dce41a9ff65/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:01.681395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd23ae4-4d2f-402a-9612-80a120c06d45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b188c61d7d1242459d7ca94e7b3438b4, user_id=e438d2ca78f54d329bbd3dce41a9ff65, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:01.684592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd23ae4-4d2f-402a-9612-80a120c06d45 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:01.697274 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1054/2110] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:30:01 2022] PUT /identity/v3/projects/b188c61d7d1242459d7ca94e7b3438b4/users/e438d2ca78f54d329bbd3dce41a9ff65/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:30:06.250343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:06.252205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:06.259107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:06.283854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119feb8-f71d-4bc6-bb6b-c051f6dcf3a3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:06.285268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119feb8-f71d-4bc6-bb6b-c051f6dcf3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:06.285985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2119feb8-f71d-4bc6-bb6b-c051f6dcf3a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:06.321416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2119feb8-f71d-4bc6-bb6b-c051f6dcf3a3 None None] MFA Rules not processed for user `e438d2ca78f54d329bbd3dce41a9ff65`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:30:06.322751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2119feb8-f71d-4bc6-bb6b-c051f6dcf3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:06.348117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcf3d0f-a9a9-4ddc-b8a7-876b2ecf7029 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:06.419753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991dc6a5-ea04-4ea8-811b-4505b14962d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:06.420518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dc6a5-ea04-4ea8-811b-4505b14962d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:06.420805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dc6a5-ea04-4ea8-811b-4505b14962d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:06.421086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991dc6a5-ea04-4ea8-811b-4505b14962d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:06.422999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-991dc6a5-ea04-4ea8-811b-4505b14962d8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:06.432569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1055/2111] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:30:06 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:30:06.446584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119feb8-f71d-4bc6-bb6b-c051f6dcf3a3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:06.448255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2119feb8-f71d-4bc6-bb6b-c051f6dcf3a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:06.495133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dc6a5-ea04-4ea8-811b-4505b14962d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:06.496646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991dc6a5-ea04-4ea8-811b-4505b14962d8 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:06.521570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2119feb8-f71d-4bc6-bb6b-c051f6dcf3a3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:06.525592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417b661f770846f189c36d0804b6dce5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:06.526366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:06.526670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:06.526941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:06.528944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1057/2112] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:30:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:30:06.621577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:06.651240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:06.653135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:06.668153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1056/2113] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:30:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:30:06.857232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6b4a1-e097-42d5-b3b0-8b95d3e0c67a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:06.857488 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6b4a1-e097-42d5-b3b0-8b95d3e0c67a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:06.857753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d6b4a1-e097-42d5-b3b0-8b95d3e0c67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:06.926335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-41d6b4a1-e097-42d5-b3b0-8b95d3e0c67a None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:30:06.927583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-41d6b4a1-e097-42d5-b3b0-8b95d3e0c67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:06.998147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:07.043528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1058/2114] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:30:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:30:07.100934 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:07.102440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:07.207596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f998f0e-3f8e-4157-9a71-5fa47771803a service glance] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:07.213706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:07.214328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:07.214602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:07.214871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:07.223333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:07.225275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:07.233391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1057/2115] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:30:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:30:17.030298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:17.230275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=47 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:17.340762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d4e5c-2b2b-420e-84c9-36c1c7f01ee8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:17.341047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d4e5c-2b2b-420e-84c9-36c1c7f01ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:17.341322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d4e5c-2b2b-420e-84c9-36c1c7f01ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:17.362423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-661d4e5c-2b2b-420e-84c9-36c1c7f01ee8 None None] MFA Rules not processed for user `f109f3d7076543d6bf16eac46a40b3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:30:17.363639 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-661d4e5c-2b2b-420e-84c9-36c1c7f01ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:17.370931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1059/2116] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:30:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:30:17.898544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:17.900244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:17.958651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba58fc4-8a11-42ba-8ca1-289f45978f3b service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:17.963943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:17.964562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:17.964959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:17.965230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:17.967234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:17.977462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:17.978914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:17.986894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1058/2117] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:30:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:30:19.860722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48b0c-401c-4a93-8372-42f41e54bf28 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:19.861383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48b0c-401c-4a93-8372-42f41e54bf28 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:19.861708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb48b0c-401c-4a93-8372-42f41e54bf28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:19.903889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dcb48b0c-401c-4a93-8372-42f41e54bf28 None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:30:19.905103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dcb48b0c-401c-4a93-8372-42f41e54bf28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:19.940179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1060/2118] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:30:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:30:19.953790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:19.998002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c878a710-ff41-4f61-a32b-c8fc91bc67de service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:20.003391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5dd3f-05d5-4458-a50e-79e0c223ba64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:20.004006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5dd3f-05d5-4458-a50e-79e0c223ba64 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:20.004290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5dd3f-05d5-4458-a50e-79e0c223ba64 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:20.004546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c5dd3f-05d5-4458-a50e-79e0c223ba64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:20.013121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5dd3f-05d5-4458-a50e-79e0c223ba64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:20.014869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c5dd3f-05d5-4458-a50e-79e0c223ba64 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:20.023709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1059/2119] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:30:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:30:21.899690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48b0c-401c-4a93-8372-42f41e54bf28 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:21.935312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb48b0c-401c-4a93-8372-42f41e54bf28 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:21.939512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:21.940144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:21.940566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:21.940837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:21.942645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:22.040995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:22.042646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:22.063681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1061/2120] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:30:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:30:25.758083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5dd3f-05d5-4458-a50e-79e0c223ba64 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:25.776997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c5dd3f-05d5-4458-a50e-79e0c223ba64 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:25.783444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:25.784161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:25.786004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:25.786337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:25.791528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:25.819289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:25.822093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:25.834302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1060/2121] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:30:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:30:32.060904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=73 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:33.650470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:30:33.652035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:33.751735 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d26e1c-87f5-4191-9b88-5a1dc29dbb46 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:30:33.756945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:30:33.757508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:30:33.757812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:30:33.758081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:30:33.759936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:30:33.772663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:30:33.774504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:30:33.785141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1062/2122] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:30:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:30:35.829209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=52 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:30:43.780875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=41 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:31:28.198239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:31:28.199986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:31:28.237788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238998b8-41d7-424a-8713-49764d7e89cb service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:31:28.241804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:31:28.242389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:31:28.242662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:31:28.242926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:31:28.244788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:31:28.326901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:31:28.328332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:31:28.347056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1061/2123] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:31:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:31:29.770812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:31:29.787446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c9b49-aab6-4f73-8d2b-602cf12d691d service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:31:29.791958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:31:29.792560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:31:29.792853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:31:29.793138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:31:29.795286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:31:29.875201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:31:29.876633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:31:29.895161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1063/2124] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:31:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:31:38.344426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=50 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:31:39.892789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=41 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:31:58.183691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:31:58.192558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068aae54-98b3-49ef-9961-7f09faecc130 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:31:58.197081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:31:58.197889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:31:58.198166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:31:58.198435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:31:58.200545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:31:58.269518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:31:58.271020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:31:58.279876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1062/2125] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:31:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:32:01.364180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:32:01.375585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8a4d42-88b4-4ac5-a48c-c993f43d5765 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:32:01.381829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:32:01.382422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:32:01.382688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:32:01.382954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:32:01.385234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:32:01.468558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:32:01.470044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:32:01.511441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1064/2126] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:32:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:32:08.275842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=28 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:32:11.515208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=35 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:33:34.095957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:33:34.106741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c993e6-4178-4cdf-86b2-f0ef6c8dd1ed service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:33:34.110873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:33:34.111489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:33:34.111770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:33:34.112047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:33:34.114097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:33:34.137393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:33:34.138852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:33:34.146449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1063/2127] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:33:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:33:44.143498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=11 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:34:41.968087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:34:41.980470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb8bd-7f54-4b94-81cc-6c86fa8cd763 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:34:41.984340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:34:41.984934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:34:41.985203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:34:41.985465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:34:41.987360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:34:42.053231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:34:42.054712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:34:42.064080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1065/2128] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:34:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:34:52.060080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=28 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:34:54.675340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:34:54.690212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdcb37a-84d5-4988-b5dc-e043b107ac2b service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:34:54.696026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:34:54.696638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:34:54.696916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:34:54.697174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:34:54.699343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:34:54.760740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:34:54.762209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:34:54.771126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1064/2129] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:34:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:04.767289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=28 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:06.519809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:06.529294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbf4e9a-d5fc-4b19-9d19-da0f4a4ad7d1 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:06.533120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8cbf-9dd2-4d52-a154-997278d5be2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:06.533788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8cbf-9dd2-4d52-a154-997278d5be2c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:06.534038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8cbf-9dd2-4d52-a154-997278d5be2c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:06.534304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9a8cbf-9dd2-4d52-a154-997278d5be2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:06.543208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a8cbf-9dd2-4d52-a154-997278d5be2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:06.544698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9a8cbf-9dd2-4d52-a154-997278d5be2c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:06.564365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:06.574520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc1392-6100-4448-a1fa-abab7c130385 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:06.576837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1066/2130] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:06.578325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e3f3a-67b5-40e7-a01d-77ea0ef44846 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:06.578947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e3f3a-67b5-40e7-a01d-77ea0ef44846 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:06.579220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e3f3a-67b5-40e7-a01d-77ea0ef44846 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:06.579486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e3f3a-67b5-40e7-a01d-77ea0ef44846 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:06.588337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e3f3a-67b5-40e7-a01d-77ea0ef44846 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:06.589901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e3f3a-67b5-40e7-a01d-77ea0ef44846 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:06.599693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1065/2131] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:16.574429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=13 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:16.595148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:37.374657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8cbf-9dd2-4d52-a154-997278d5be2c service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:37.386469 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9a8cbf-9dd2-4d52-a154-997278d5be2c service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:37.391751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfec6ab-0ddf-433e-9adc-be480ddb533c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:37.392355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfec6ab-0ddf-433e-9adc-be480ddb533c service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:37.392630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfec6ab-0ddf-433e-9adc-be480ddb533c service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:37.392897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfec6ab-0ddf-433e-9adc-be480ddb533c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:37.401477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfec6ab-0ddf-433e-9adc-be480ddb533c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:37.403106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfec6ab-0ddf-433e-9adc-be480ddb533c service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:37.414070 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1067/2132] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:38.429400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e3f3a-67b5-40e7-a01d-77ea0ef44846 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:38.438895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e3f3a-67b5-40e7-a01d-77ea0ef44846 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:38.442380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:38.443001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:38.443273 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:38.443533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:38.454068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:38.455926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:38.467141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1066/2133] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:39.667826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfec6ab-0ddf-433e-9adc-be480ddb533c service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:39.680024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfec6ab-0ddf-433e-9adc-be480ddb533c service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:39.684763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe3e1-3ace-4239-9980-67369c9de34a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:39.685381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe3e1-3ace-4239-9980-67369c9de34a service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:39.685698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe3e1-3ace-4239-9980-67369c9de34a service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:39.686126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7fe3e1-3ace-4239-9980-67369c9de34a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:39.708238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fe3e1-3ace-4239-9980-67369c9de34a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:39.709793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7fe3e1-3ace-4239-9980-67369c9de34a service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:39.721323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1068/2134] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:40.320625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:40.322510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:40.340215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d1f7e4-726d-4309-9771-3ec64082f2c0 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:40.348204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:40.348853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:40.349122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:40.349468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:40.353180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:40.373757 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:40.375493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:40.403866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1067/2135] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:44.528615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe3e1-3ace-4239-9980-67369c9de34a service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:44.538717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7fe3e1-3ace-4239-9980-67369c9de34a service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:44.543022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:44.543626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:44.543902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:44.544165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:44.546306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:44.642303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:44.643763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:44.664958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1069/2136] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:45.130518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:45.140719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c7ca67-ddd0-4336-99cb-ed2534887d6e service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:45.144925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:45.145776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:45.146049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:45.146314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:45.148207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:45.244986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:45.246445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:45.268221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1068/2137] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:45 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:54.408665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:54.417354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c96055-506c-46e0-9865-f33126325a0c service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:54.421012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119974b-8009-4c83-b2e8-c9f226228f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:54.421609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119974b-8009-4c83-b2e8-c9f226228f1b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:54.421916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119974b-8009-4c83-b2e8-c9f226228f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:54.422180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119974b-8009-4c83-b2e8-c9f226228f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:54.423928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c119974b-8009-4c83-b2e8-c9f226228f1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:54.485995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119974b-8009-4c83-b2e8-c9f226228f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:54.487421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119974b-8009-4c83-b2e8-c9f226228f1b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:54.497227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1070/2138] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:55.264223 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=55 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:55.555408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:55.557923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:55.670286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda9709b-c75b-4b5d-bc10-7a8e3bec49be service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:55.675113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:55.675748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:55.676045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:55.676317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] PATH_INFO: `/v3/users/fcc151de897b4297898677a651cb0723` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:55.677434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcc151de897b4297898677a651cb0723)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:55.679415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:55.738297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcc151de897b4297898677a651cb0723'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:55.745088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcc151de897b4297898677a651cb0723'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:55.745818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcc151de897b4297898677a651cb0723'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:55.754082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcc151de897b4297898677a651cb0723'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:55.765913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcc151de897b4297898677a651cb0723'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:55.771917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1069/2139] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:35:55 2022] DELETE /identity/v3/users/fcc151de897b4297898677a651cb0723 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:35:55.776692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8807-f85d-4c73-9650-9492809e8aa3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:55.777163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8807-f85d-4c73-9650-9492809e8aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:55.777257 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde8807-f85d-4c73-9650-9492809e8aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:55.806560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2fde8807-f85d-4c73-9650-9492809e8aa3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:35:55.807714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2fde8807-f85d-4c73-9650-9492809e8aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:55.828142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:55.868580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1071/2140] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:35:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:35:55.878812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:55.880347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:55.939592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51addc9c-68cb-48a5-aae9-cfb854da2c31 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:55.944215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5516c1-0e60-4ae0-8197-3d8cfbb19c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:55.944826 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5516c1-0e60-4ae0-8197-3d8cfbb19c7e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:55.945101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5516c1-0e60-4ae0-8197-3d8cfbb19c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:55.945362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5516c1-0e60-4ae0-8197-3d8cfbb19c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:55.951812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:55.956009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5516c1-0e60-4ae0-8197-3d8cfbb19c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:55.958069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5516c1-0e60-4ae0-8197-3d8cfbb19c7e service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:55.968214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1070/2141] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:35:56.238462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde8807-f85d-4c73-9650-9492809e8aa3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:56.239906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2fde8807-f85d-4c73-9650-9492809e8aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:56.253185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde8807-f85d-4c73-9650-9492809e8aa3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:56.259025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:56.259748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:56.260090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:56.260395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] PATH_INFO: `/v3/projects/db540aeb30d64a73856441da6a1e0431` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:56.261714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db540aeb30d64a73856441da6a1e0431)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:56.263721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:56.312835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db540aeb30d64a73856441da6a1e0431', 'request_id': 'req-d27da046-c620-4f86-b44a-d457ca8c01e0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.318486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db540aeb30d64a73856441da6a1e0431', 'request_id': 'req-d27da046-c620-4f86-b44a-d457ca8c01e0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.324644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db540aeb30d64a73856441da6a1e0431', 'request_id': 'req-d27da046-c620-4f86-b44a-d457ca8c01e0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.339926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] The token cache is being invalidate because project db540aeb30d64a73856441da6a1e0431 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:35:56.340208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.340917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.342594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1072/2142] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:35:56 2022] DELETE /identity/v3/projects/db540aeb30d64a73856441da6a1e0431 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:35:56.347110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5516c1-0e60-4ae0-8197-3d8cfbb19c7e service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:56.357098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5516c1-0e60-4ae0-8197-3d8cfbb19c7e service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:56.415599 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:56.416240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:56.416515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:56.416783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] PATH_INFO: `/v3/users/ac92924b370b4e25afce97be0fd29f3f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:56.417950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac92924b370b4e25afce97be0fd29f3f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:56.419876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:56.471627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac92924b370b4e25afce97be0fd29f3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.478565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac92924b370b4e25afce97be0fd29f3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.479316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac92924b370b4e25afce97be0fd29f3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.501856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac92924b370b4e25afce97be0fd29f3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.504279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-186e1849-de7e-4b2a-a8dd-6bceefc17281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac92924b370b4e25afce97be0fd29f3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.514704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1071/2143] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:35:56 2022] DELETE /identity/v3/users/ac92924b370b4e25afce97be0fd29f3f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:35:56.817864 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:56.817864 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:56.834244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27da046-c620-4f86-b44a-d457ca8c01e0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:56.907417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:56.908368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:56.908783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:56.909186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] PATH_INFO: `/v3/projects/77f8523343b242018ffbc0e83569e093` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:56.910861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77f8523343b242018ffbc0e83569e093)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:56.914008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:56.981090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77f8523343b242018ffbc0e83569e093', 'request_id': 'req-098109e9-945b-4f1d-ad67-1b8b858c8893'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.983206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:56.988862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77f8523343b242018ffbc0e83569e093', 'request_id': 'req-098109e9-945b-4f1d-ad67-1b8b858c8893'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:56.997665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77f8523343b242018ffbc0e83569e093', 'request_id': 'req-098109e9-945b-4f1d-ad67-1b8b858c8893'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:57.015541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] The token cache is being invalidate because project 77f8523343b242018ffbc0e83569e093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:35:57.015953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:57.016980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-098109e9-945b-4f1d-ad67-1b8b858c8893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:35:57.018574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1073/2144] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:35:56 2022] DELETE /identity/v3/projects/77f8523343b242018ffbc0e83569e093 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:35:57.024486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbeb6c-c442-4a97-98f2-844f9bd9632a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.024767 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbeb6c-c442-4a97-98f2-844f9bd9632a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.025034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bbeb6c-c442-4a97-98f2-844f9bd9632a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.075474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c5bbeb6c-c442-4a97-98f2-844f9bd9632a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:35:57.076695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c5bbeb6c-c442-4a97-98f2-844f9bd9632a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:57.122745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:57.139945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1072/2145] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:35:57.147990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077461e-d2dc-45a9-9c8a-dea79cc43853 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.148271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077461e-d2dc-45a9-9c8a-dea79cc43853 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.148541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e077461e-d2dc-45a9-9c8a-dea79cc43853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.168968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e077461e-d2dc-45a9-9c8a-dea79cc43853 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:35:57.170139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e077461e-d2dc-45a9-9c8a-dea79cc43853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:57.179660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1074/2146] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:35:57.183898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbeb6c-c442-4a97-98f2-844f9bd9632a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:57.194214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bbeb6c-c442-4a97-98f2-844f9bd9632a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:57.199349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:57.199950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.200227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.200505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.201151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:35:57.201452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:57.203098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:57.215640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1073/2147] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:35:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:35:57.220457 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077461e-d2dc-45a9-9c8a-dea79cc43853 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:57.231114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e077461e-d2dc-45a9-9c8a-dea79cc43853 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:57.237942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e32947-22e9-4194-9f07-f9469b70f967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:57.238601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e32947-22e9-4194-9f07-f9469b70f967 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.238880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e32947-22e9-4194-9f07-f9469b70f967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.239209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e32947-22e9-4194-9f07-f9469b70f967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.239906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e32947-22e9-4194-9f07-f9469b70f967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:57.241866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e32947-22e9-4194-9f07-f9469b70f967 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:57.287834 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1075/2148] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:35:57 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:35:57.294237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:57.306481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0902ab-9ac1-474d-adec-75a9c18de66f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:57.372105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17c731-48d0-400b-86f2-ad2945f3d8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:57.372853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17c731-48d0-400b-86f2-ad2945f3d8d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.372990 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17c731-48d0-400b-86f2-ad2945f3d8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.373265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17c731-48d0-400b-86f2-ad2945f3d8d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.373940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17c731-48d0-400b-86f2-ad2945f3d8d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:57.375916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17c731-48d0-400b-86f2-ad2945f3d8d4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:57.402312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1074/2149] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:35:57 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:35:57.406704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e32947-22e9-4194-9f07-f9469b70f967 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:57.417959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e32947-22e9-4194-9f07-f9469b70f967 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:57.423027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71550c-138d-4510-9311-2f417c6e646f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:57.423648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71550c-138d-4510-9311-2f417c6e646f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.423936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71550c-138d-4510-9311-2f417c6e646f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.424204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da71550c-138d-4510-9311-2f417c6e646f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.424954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71550c-138d-4510-9311-2f417c6e646f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:57.426745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da71550c-138d-4510-9311-2f417c6e646f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:57.439532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1076/2150] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:35:57.443671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17c731-48d0-400b-86f2-ad2945f3d8d4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:57.454194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17c731-48d0-400b-86f2-ad2945f3d8d4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:57.459306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3f2ae-7b5b-4469-9718-fdc6bb1a52e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:57.459924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3f2ae-7b5b-4469-9718-fdc6bb1a52e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.460183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3f2ae-7b5b-4469-9718-fdc6bb1a52e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.460456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3f2ae-7b5b-4469-9718-fdc6bb1a52e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.461133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3f2ae-7b5b-4469-9718-fdc6bb1a52e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:57.462754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3f2ae-7b5b-4469-9718-fdc6bb1a52e8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:57.474827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1075/2151] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:35:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:35:57.479332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71550c-138d-4510-9311-2f417c6e646f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:57.490391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da71550c-138d-4510-9311-2f417c6e646f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:57.495812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:57.496551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.496895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.497224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] PATH_INFO: `/v3/projects/04eff040818b4f2eb4fe7f7e9dbc8460/users/70276ea5068f4bfc85d4216aca67fb1d/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.514356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] RBAC: Authorizing `identity:create_grant(project_id=04eff040818b4f2eb4fe7f7e9dbc8460, user_id=70276ea5068f4bfc85d4216aca67fb1d, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:57.517184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:57.532371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1077/2152] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:35:57 2022] PUT /identity/v3/projects/04eff040818b4f2eb4fe7f7e9dbc8460/users/70276ea5068f4bfc85d4216aca67fb1d/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:35:57.537861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091bec9-8026-4dea-858c-03a8db3af5c1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.538144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091bec9-8026-4dea-858c-03a8db3af5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.538414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091bec9-8026-4dea-858c-03a8db3af5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.564872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3091bec9-8026-4dea-858c-03a8db3af5c1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:35:57.566736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3091bec9-8026-4dea-858c-03a8db3af5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:57.626135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1076/2153] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:35:57.639283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:57.641098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:57.699959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e9be34-131e-4b47-9608-d2cf7be67abb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:57.704161 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc0378-fb34-4e2e-afc3-1e9a71a55ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:57.704775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc0378-fb34-4e2e-afc3-1e9a71a55ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:57.705056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc0378-fb34-4e2e-afc3-1e9a71a55ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:57.705324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc0378-fb34-4e2e-afc3-1e9a71a55ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:57.714907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc0378-fb34-4e2e-afc3-1e9a71a55ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:57.716356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc0378-fb34-4e2e-afc3-1e9a71a55ef1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:57.726494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1078/2154] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:35:58.348724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5d18-2b06-48df-9c05-58c0c9a65c8c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:58.348999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5d18-2b06-48df-9c05-58c0c9a65c8c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:58.349263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cf5d18-2b06-48df-9c05-58c0c9a65c8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:58.378205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-87cf5d18-2b06-48df-9c05-58c0c9a65c8c None None] MFA Rules not processed for user `70276ea5068f4bfc85d4216aca67fb1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:35:58.379518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-87cf5d18-2b06-48df-9c05-58c0c9a65c8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:58.425428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:58.447482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1077/2155] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:35:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:35:58.452141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc0378-fb34-4e2e-afc3-1e9a71a55ef1 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:58.462053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc0378-fb34-4e2e-afc3-1e9a71a55ef1 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:58.466852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da426f4-908c-412d-ac58-6d379c5b806b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:58.467536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da426f4-908c-412d-ac58-6d379c5b806b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:58.467878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da426f4-908c-412d-ac58-6d379c5b806b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:58.468212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da426f4-908c-412d-ac58-6d379c5b806b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:58.468932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da426f4-908c-412d-ac58-6d379c5b806b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:58.471100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da426f4-908c-412d-ac58-6d379c5b806b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:58.503106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1079/2156] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:35:58 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:35:58.507903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf5d18-2b06-48df-9c05-58c0c9a65c8c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:58.521991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cf5d18-2b06-48df-9c05-58c0c9a65c8c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:58.592774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6fd46-9877-41a3-8472-874f971e41f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:58.593418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6fd46-9877-41a3-8472-874f971e41f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:58.593756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6fd46-9877-41a3-8472-874f971e41f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:58.594285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6fd46-9877-41a3-8472-874f971e41f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:58.594999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6fd46-9877-41a3-8472-874f971e41f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:58.597299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6fd46-9877-41a3-8472-874f971e41f1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:58.627619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1078/2157] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:35:58 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:35:58.631875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da426f4-908c-412d-ac58-6d379c5b806b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:58.642495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da426f4-908c-412d-ac58-6d379c5b806b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:58.647303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a2b5-a272-43ce-aaab-ddafa7c0260d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:58.648018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a2b5-a272-43ce-aaab-ddafa7c0260d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:58.648361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a2b5-a272-43ce-aaab-ddafa7c0260d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:58.648693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a278a2b5-a272-43ce-aaab-ddafa7c0260d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:58.649422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278a2b5-a272-43ce-aaab-ddafa7c0260d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:58.651124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a278a2b5-a272-43ce-aaab-ddafa7c0260d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:58.662140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1080/2158] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:35:58.666099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6fd46-9877-41a3-8472-874f971e41f1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:58.676080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6fd46-9877-41a3-8472-874f971e41f1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:58.680792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a789099-7f25-4581-a81a-664bd007a023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:58.681570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a789099-7f25-4581-a81a-664bd007a023 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:58.681955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a789099-7f25-4581-a81a-664bd007a023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:58.682228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a789099-7f25-4581-a81a-664bd007a023 admin admin] PATH_INFO: `/v3/projects/a1dde76be40c49539db2c4ed29d0cd04/users/3be9f4e2773641a695d7e8ab11174c9b/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:58.696520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a789099-7f25-4581-a81a-664bd007a023 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1dde76be40c49539db2c4ed29d0cd04, user_id=3be9f4e2773641a695d7e8ab11174c9b, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:58.699190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a789099-7f25-4581-a81a-664bd007a023 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:58.712970 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1079/2159] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:35:58 2022] PUT /identity/v3/projects/a1dde76be40c49539db2c4ed29d0cd04/users/3be9f4e2773641a695d7e8ab11174c9b/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:35:58.716869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a2b5-a272-43ce-aaab-ddafa7c0260d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:58.727907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a278a2b5-a272-43ce-aaab-ddafa7c0260d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:58.767888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:58.795332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2f0728-d4e8-4f05-9f62-306216d03561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:58.796263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0728-d4e8-4f05-9f62-306216d03561 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:58.796648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0728-d4e8-4f05-9f62-306216d03561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:58.797044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2f0728-d4e8-4f05-9f62-306216d03561 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:58.798196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f0728-d4e8-4f05-9f62-306216d03561 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:58.799818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2f0728-d4e8-4f05-9f62-306216d03561 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:58.810295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1081/2160] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:35:58.814655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a789099-7f25-4581-a81a-664bd007a023 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:58.825623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a789099-7f25-4581-a81a-664bd007a023 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:58.831817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:58.832759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:58.833054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:58.833380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] PATH_INFO: `/v3/projects/a1dde76be40c49539db2c4ed29d0cd04/users/3be9f4e2773641a695d7e8ab11174c9b/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:58.836479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1dde76be40c49539db2c4ed29d0cd04, user_id=3be9f4e2773641a695d7e8ab11174c9b, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:58.840519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:58.857898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1080/2161] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:35:58 2022] PUT /identity/v3/projects/a1dde76be40c49539db2c4ed29d0cd04/users/3be9f4e2773641a695d7e8ab11174c9b/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:35:59.513126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7742ea-f449-4daf-a44f-1154842a8531 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:59.513702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7742ea-f449-4daf-a44f-1154842a8531 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:59.513702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7742ea-f449-4daf-a44f-1154842a8531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:59.542967 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3c7742ea-f449-4daf-a44f-1154842a8531 None None] MFA Rules not processed for user `3be9f4e2773641a695d7e8ab11174c9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:35:59.544204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c7742ea-f449-4daf-a44f-1154842a8531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:59.626083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1082/2162] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:35:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:35:59.634808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:35:59.636704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:35:59.695763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc23bef0-be9c-4e98-8f45-c1a05cc3985b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:35:59.701404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:35:59.702064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:35:59.702339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:35:59.702608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:35:59.751575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:35:59.761560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:35:59.763714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:35:59.776012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1081/2163] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:35:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:36:09.618400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=46 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:36:09.768801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:36:28.263206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7742ea-f449-4daf-a44f-1154842a8531 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:36:28.282944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7742ea-f449-4daf-a44f-1154842a8531 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:36:28.287623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:36:28.288250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:36:28.288525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:36:28.288788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:36:28.291009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:36:28.354408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:36:28.355959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:36:28.365155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1083/2164] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:36:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:36:28.744420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:36:28.746828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:36:28.763808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee570152-3914-4b04-a157-e8d7e802459c service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:36:28.770131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:36:28.770916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:36:28.771283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:36:28.771652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:36:28.774449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:36:28.833977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:36:28.835426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:36:28.844324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1082/2165] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:36:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:36:38.361188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=28 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:36:38.840498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=28 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:36:58.387610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:36:58.396934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3121f8b5-c5a1-49a5-b949-1a4ad4c5cc91 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:36:58.401307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:36:58.401960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:36:58.402432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:36:58.402522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:36:58.404525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:36:58.486692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:36:58.488148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:36:58.506264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1084/2166] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:36:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:37:02.275446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:37:02.284668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8213e7-cde3-4bdb-99bc-136d28566e79 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:37:02.289204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:37:02.289886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:37:02.290185 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:37:02.290569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:37:02.292550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:37:02.369834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:37:02.371254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:37:02.380182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1083/2167] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:37:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:37:08.503312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=41 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:37:12.376092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=33 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:38:39.067780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:38:39.069247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:38:39.125260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb9d893-4000-483a-b9bf-2b2092636b3a service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:38:39.129223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:38:39.129862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:38:39.130132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:38:39.130394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:38:39.132164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:38:39.142152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:38:39.143573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:38:39.151435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1085/2168] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:38:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:38:49.147857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=26 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:39:42.967618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:39:42.976838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bb2b5d-0009-4945-bf32-9714287aaa5e service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:39:42.981438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:39:42.982123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:39:42.982337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:39:42.982597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:39:42.984595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:39:43.071904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:39:43.074276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:39:43.096887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1084/2169] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:39:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:39:53.094513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=41 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:06.047507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:06.057756 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b16472-d8cf-4151-a277-42685a4bcbda service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:06.062305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:06.062779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:06.063224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:06.063224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:06.063476 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:06.065680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:06.073026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15daf019-1116-4403-bc84-b49144c05355 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:06.077107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:06.077722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:06.078004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:06.078272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:06.080130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:06.151858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:06.152843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:06.153518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:06.154368 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:06.162507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1085/2170] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:06 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:40:06.173725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1086/2171] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:40:07.433065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:07.443558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebeabed-356c-48fc-9f6e-20fd199e5e18 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:07.447136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:07.447745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:07.448018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:07.448288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:07.450172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:07.520745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:07.522251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:07.535394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1086/2172] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:40:10.855884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:10.857339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:10.935911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:10.937478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:10.947654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c173b6d-6326-42d1-b1ae-7a07e62b960e service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:10.951262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4be7cf-194b-4035-9fef-69a1a3439366 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:10.953333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:10.954249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:10.954955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:10.955430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:10.955833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:10.956461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:10.956889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:10.957322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:10.958362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] PATH_INFO: `/v3/users/bad6116fbd5c407c9f44084fadd9531e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:10.959531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bad6116fbd5c407c9f44084fadd9531e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:10.961416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:10.975493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:10.980514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:10.991539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1087/2173] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:40:11.018875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bad6116fbd5c407c9f44084fadd9531e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.027158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bad6116fbd5c407c9f44084fadd9531e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.027995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bad6116fbd5c407c9f44084fadd9531e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.036568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bad6116fbd5c407c9f44084fadd9531e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.047914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:11.050338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f87ad32f-041c-40e5-8f32-dd8fde905a3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bad6116fbd5c407c9f44084fadd9531e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.057861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1087/2174] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:40:10 2022] DELETE /identity/v3/users/bad6116fbd5c407c9f44084fadd9531e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:40:11.242449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:11.258753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24cde5b-b4b4-4116-a40d-64ad0e102d49 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:11.321388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:11.322062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.322337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.322605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] PATH_INFO: `/v3/projects/6bd3e20c75e94f3a9c17fcb3029bfdff` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.323814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd3e20c75e94f3a9c17fcb3029bfdff)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:11.325785 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:11.327306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:11.382472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'request_id': 'req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.388357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'request_id': 'req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.394980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd3e20c75e94f3a9c17fcb3029bfdff', 'request_id': 'req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.410115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] The token cache is being invalidate because project 6bd3e20c75e94f3a9c17fcb3029bfdff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:40:11.410406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.411029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c1cb2e29-c361-4ea0-a1e5-d4dcc363d5c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:40:11.412579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1088/2175] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:40:11 2022] DELETE /identity/v3/projects/6bd3e20c75e94f3a9c17fcb3029bfdff => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:40:11.419029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a058cba-db07-4d11-9ab1-e1b17c95e221 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.419314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a058cba-db07-4d11-9ab1-e1b17c95e221 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.419580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a058cba-db07-4d11-9ab1-e1b17c95e221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.440268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7a058cba-db07-4d11-9ab1-e1b17c95e221 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:40:11.441475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a058cba-db07-4d11-9ab1-e1b17c95e221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:11.503502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1088/2176] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:40:11.511550 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7803b4-04b5-4b07-bab3-8bf2bcb7c1ac None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.511820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7803b4-04b5-4b07-bab3-8bf2bcb7c1ac None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.512088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7803b4-04b5-4b07-bab3-8bf2bcb7c1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.538601 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cc7803b4-04b5-4b07-bab3-8bf2bcb7c1ac None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:40:11.539739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cc7803b4-04b5-4b07-bab3-8bf2bcb7c1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:11.548179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1089/2177] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:40:11.552577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a058cba-db07-4d11-9ab1-e1b17c95e221 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:11.568907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a058cba-db07-4d11-9ab1-e1b17c95e221 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:11.574482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:11.575113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.575388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.575654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.576310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:40:11.576615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:11.578250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:11.591087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1089/2178] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:40:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:40:11.595986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7803b4-04b5-4b07-bab3-8bf2bcb7c1ac None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:11.605498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7803b4-04b5-4b07-bab3-8bf2bcb7c1ac None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:11.610230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2026f0b-b496-4065-8594-37ac812af37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:11.610849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026f0b-b496-4065-8594-37ac812af37f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.611127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026f0b-b496-4065-8594-37ac812af37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.611398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2026f0b-b496-4065-8594-37ac812af37f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.612075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2026f0b-b496-4065-8594-37ac812af37f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:11.614065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2026f0b-b496-4065-8594-37ac812af37f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:11.643179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1090/2179] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:40:11 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:40:11.647384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:11.657347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825ced9-1af0-4179-80f7-2a57b953cfa4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:11.717397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccdf1e-3a95-43bd-8649-02f434e26ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:11.718064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccdf1e-3a95-43bd-8649-02f434e26ef6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.718710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccdf1e-3a95-43bd-8649-02f434e26ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.718979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eccdf1e-3a95-43bd-8649-02f434e26ef6 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.719621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccdf1e-3a95-43bd-8649-02f434e26ef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:11.721867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eccdf1e-3a95-43bd-8649-02f434e26ef6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:11.747863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1090/2180] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:40:11 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:40:11.749384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2026f0b-b496-4065-8594-37ac812af37f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:11.760031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2026f0b-b496-4065-8594-37ac812af37f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:11.765703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac026bc-75cf-4c07-8529-a71da245670f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:11.766331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac026bc-75cf-4c07-8529-a71da245670f admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.766610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac026bc-75cf-4c07-8529-a71da245670f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.766877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac026bc-75cf-4c07-8529-a71da245670f admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.767513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac026bc-75cf-4c07-8529-a71da245670f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:11.769120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac026bc-75cf-4c07-8529-a71da245670f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:11.785811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1091/2181] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:40:11.785930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccdf1e-3a95-43bd-8649-02f434e26ef6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:11.794065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eccdf1e-3a95-43bd-8649-02f434e26ef6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:11.798798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79e303-c2f4-48de-890c-ed062f32c1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:11.799437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e303-c2f4-48de-890c-ed062f32c1c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.799711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e303-c2f4-48de-890c-ed062f32c1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.799968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79e303-c2f4-48de-890c-ed062f32c1c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.800618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79e303-c2f4-48de-890c-ed062f32c1c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:11.802298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79e303-c2f4-48de-890c-ed062f32c1c5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:11.813149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1091/2182] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:40:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:40:11.817485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac026bc-75cf-4c07-8529-a71da245670f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:11.828202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac026bc-75cf-4c07-8529-a71da245670f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:11.836839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:11.837510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.837829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.838101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] PATH_INFO: `/v3/projects/ff81348c32db4e40a8e331317d14e3d2/users/7de0d01f891d4d73bc44810aacbca477/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.853215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff81348c32db4e40a8e331317d14e3d2, user_id=7de0d01f891d4d73bc44810aacbca477, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:11.855926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:11.870210 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1092/2183] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:40:11 2022] PUT /identity/v3/projects/ff81348c32db4e40a8e331317d14e3d2/users/7de0d01f891d4d73bc44810aacbca477/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:40:11.875399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f8659-2218-4109-b672-7959f10b351a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:11.875681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f8659-2218-4109-b672-7959f10b351a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:11.875953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f8659-2218-4109-b672-7959f10b351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:11.895964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-990f8659-2218-4109-b672-7959f10b351a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:40:11.897180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-990f8659-2218-4109-b672-7959f10b351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:11.951976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:11.963173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1092/2184] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:40:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:40:11.973474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:11.975291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:12.035627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96ee77-2218-4fc4-a73f-7d1851677b4a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:12.040366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:12.040981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:12.041256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:12.041528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:12.051949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:12.053335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:12.062918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1093/2185] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:40:16.308453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65a2c-2b25-4ff9-978f-636d0b391baf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:16.308950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65a2c-2b25-4ff9-978f-636d0b391baf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:16.309041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf65a2c-2b25-4ff9-978f-636d0b391baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:16.341244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ebf65a2c-2b25-4ff9-978f-636d0b391baf None None] MFA Rules not processed for user `7de0d01f891d4d73bc44810aacbca477`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:40:16.342469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ebf65a2c-2b25-4ff9-978f-636d0b391baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:16.419404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1093/2186] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:40:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:40:16.433760 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:16.435470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:16.470922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:16.519867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2dbf3-7945-4b6a-976f-7cac9a55ac2b service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:16.525619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f41c4-fdb9-411f-bc18-85c3f973aa36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:16.526334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f41c4-fdb9-411f-bc18-85c3f973aa36 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:16.526613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f41c4-fdb9-411f-bc18-85c3f973aa36 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:16.526880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65f41c4-fdb9-411f-bc18-85c3f973aa36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:16.535617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f41c4-fdb9-411f-bc18-85c3f973aa36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:16.537237 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65f41c4-fdb9-411f-bc18-85c3f973aa36 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:16.546544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1094/2187] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:16 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:40:17.806312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a4f54-8e46-4ed7-b75d-0a2747e87099 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:17.809064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a4f54-8e46-4ed7-b75d-0a2747e87099 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:17.809486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18a4f54-8e46-4ed7-b75d-0a2747e87099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:17.831137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e18a4f54-8e46-4ed7-b75d-0a2747e87099 None None] MFA Rules not processed for user `7de0d01f891d4d73bc44810aacbca477`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:40:17.832304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e18a4f54-8e46-4ed7-b75d-0a2747e87099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:17.839456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1094/2188] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:40:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:40:18.189810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f41c4-fdb9-411f-bc18-85c3f973aa36 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:18.200624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65f41c4-fdb9-411f-bc18-85c3f973aa36 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:18.204569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:18.205268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:18.205706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:18.205978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:18.207783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:18.217668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:18.219474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:18.230688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1095/2189] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:40:18.586539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a4f54-8e46-4ed7-b75d-0a2747e87099 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:18.588114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e18a4f54-8e46-4ed7-b75d-0a2747e87099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:18.677182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18a4f54-8e46-4ed7-b75d-0a2747e87099 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:18.682893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:18.684260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:18.684260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:18.684507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:18.686590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:18.701448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:18.703915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:18.715593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1095/2190] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:40:26.704027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:26.810780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e50161-7f08-4c5f-866c-dfae0924b03d service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:26.816550 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:26.817234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:26.817597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:26.817963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:26.819779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:26.855310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:26.856958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:26.869826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1096/2191] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:40:28.710393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=67 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:36.863721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=66 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:40:45.400372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:40:45.402279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:45.499533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6843bab6-1b2c-4431-b20f-bd64548500b1 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:40:45.505146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:40:45.505953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:40:45.506318 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:40:45.506661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:40:45.508791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:40:45.520756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:40:45.522364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:40:45.532024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1096/2192] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:40:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:40:55.527937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=37 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:00.497380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:00.499053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:00.582386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735398e-9814-4342-a00d-629eed4bcfa2 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:00.586526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:00.587176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:00.587448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:00.587713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:00.589509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:00.649507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:00.651061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:00.659729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1097/2193] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:10.656073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=48 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:28.374798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:28.376427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:28.410252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0ec27-4210-4065-993f-edc2f5e8ca93 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:28.414190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:28.414846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:28.415119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:28.415393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:28.417292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:28.500004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:28.501357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:28.520035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1097/2194] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:28 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:29.557288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:29.576993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d1a17-4ee3-46aa-9d98-83c992eb4c0c service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:29.581115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621a4f8-3e9b-477e-841a-1ea4be9b9beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:29.581746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621a4f8-3e9b-477e-841a-1ea4be9b9beb service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:29.582023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621a4f8-3e9b-477e-841a-1ea4be9b9beb service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:29.582419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621a4f8-3e9b-477e-841a-1ea4be9b9beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:29.584315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0621a4f8-3e9b-477e-841a-1ea4be9b9beb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:29.644341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621a4f8-3e9b-477e-841a-1ea4be9b9beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:29.645816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621a4f8-3e9b-477e-841a-1ea4be9b9beb service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:29.658156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1098/2195] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:38.517728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=50 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:39.651009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=32 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:43.270397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:43.271975 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:43.286090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbce3e-d624-4726-b4c4-f0b45f2c3b36 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:43.291052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:43.291701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:43.292050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:43.292357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] PATH_INFO: `/v3/users/c6b2448558194fbfb02e3e76bad02b06` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:43.293510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b2448558194fbfb02e3e76bad02b06)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:43.295461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:43.344419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b2448558194fbfb02e3e76bad02b06'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.351383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b2448558194fbfb02e3e76bad02b06'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.352052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b2448558194fbfb02e3e76bad02b06'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.360606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b2448558194fbfb02e3e76bad02b06'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.372552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b2448558194fbfb02e3e76bad02b06'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.378521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1098/2196] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:41:43 2022] DELETE /identity/v3/users/c6b2448558194fbfb02e3e76bad02b06 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:43.383459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cb644-8a65-4e81-9248-3c93097ed28b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:43.383972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cb644-8a65-4e81-9248-3c93097ed28b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:43.383972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406cb644-8a65-4e81-9248-3c93097ed28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:43.404422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-406cb644-8a65-4e81-9248-3c93097ed28b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:43.405733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-406cb644-8a65-4e81-9248-3c93097ed28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:43.479260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1099/2197] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:43.491630 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:43.493167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:43.564255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26deee1-c534-4f2a-b47c-b27efddac766 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:43.567899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf791e-530c-46de-bb4a-862ac034ed52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:43.568506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf791e-530c-46de-bb4a-862ac034ed52 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:43.568779 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf791e-530c-46de-bb4a-862ac034ed52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:43.569045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf791e-530c-46de-bb4a-862ac034ed52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:43.577781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf791e-530c-46de-bb4a-862ac034ed52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:43.579235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf791e-530c-46de-bb4a-862ac034ed52 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:43.588296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1099/2198] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:43.885007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cb644-8a65-4e81-9248-3c93097ed28b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:43.886251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-406cb644-8a65-4e81-9248-3c93097ed28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:43.898524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406cb644-8a65-4e81-9248-3c93097ed28b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:43.905306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:43.905969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:43.906338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:43.906665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] PATH_INFO: `/v3/projects/edc99ad3772145ce808197eaf98cb5a3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:43.907948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc99ad3772145ce808197eaf98cb5a3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:43.909964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:43.959898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc99ad3772145ce808197eaf98cb5a3', 'request_id': 'req-b31849d4-1a11-481c-bfa1-721f309e5ed0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.965437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc99ad3772145ce808197eaf98cb5a3', 'request_id': 'req-b31849d4-1a11-481c-bfa1-721f309e5ed0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.972019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc99ad3772145ce808197eaf98cb5a3', 'request_id': 'req-b31849d4-1a11-481c-bfa1-721f309e5ed0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.985233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] The token cache is being invalidate because project edc99ad3772145ce808197eaf98cb5a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:41:43.985497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.986190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b31849d4-1a11-481c-bfa1-721f309e5ed0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:43.987700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1100/2199] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:41:43 2022] DELETE /identity/v3/projects/edc99ad3772145ce808197eaf98cb5a3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:43.993705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3243c-b2ce-4bb0-9508-65a6a7fc994d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:43.993988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3243c-b2ce-4bb0-9508-65a6a7fc994d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:43.994253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3243c-b2ce-4bb0-9508-65a6a7fc994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.015821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-afe3243c-b2ce-4bb0-9508-65a6a7fc994d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:44.017026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-afe3243c-b2ce-4bb0-9508-65a6a7fc994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:44.075827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1100/2200] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:44.086794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa27b1-f1b0-4408-baa1-75ef6dd80b27 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.087074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa27b1-f1b0-4408-baa1-75ef6dd80b27 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.087339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa27b1-f1b0-4408-baa1-75ef6dd80b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.108954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-72aa27b1-f1b0-4408-baa1-75ef6dd80b27 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:44.110308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-72aa27b1-f1b0-4408-baa1-75ef6dd80b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:44.119570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1101/2201] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:44.124170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3243c-b2ce-4bb0-9508-65a6a7fc994d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.134400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3243c-b2ce-4bb0-9508-65a6a7fc994d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:44.139106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:44.139807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.140083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.140331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.141052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:41:44.141352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:44.142961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:44.144430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:44.156177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1101/2202] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:41:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:41:44.160696 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa27b1-f1b0-4408-baa1-75ef6dd80b27 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.169914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa27b1-f1b0-4408-baa1-75ef6dd80b27 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:44.176847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681df6cb-6f0d-401f-bb88-3274657bf4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:44.177453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681df6cb-6f0d-401f-bb88-3274657bf4ef admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.177763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681df6cb-6f0d-401f-bb88-3274657bf4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.178028 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681df6cb-6f0d-401f-bb88-3274657bf4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.178671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681df6cb-6f0d-401f-bb88-3274657bf4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:44.180579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681df6cb-6f0d-401f-bb88-3274657bf4ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:44.209163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1102/2203] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:41:44 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:41:44.213451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.224147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7aeb3-6dc8-4b29-b320-0edec959b753 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:44.288276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3141026-5134-4645-99a6-6a22d838cd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:44.289075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3141026-5134-4645-99a6-6a22d838cd29 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.289352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3141026-5134-4645-99a6-6a22d838cd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.289679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3141026-5134-4645-99a6-6a22d838cd29 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.290332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3141026-5134-4645-99a6-6a22d838cd29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:44.292223 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3141026-5134-4645-99a6-6a22d838cd29 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:44.318131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1102/2204] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:41:44 2022] POST /identity/v3/users => generated 286 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:41:44.319676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681df6cb-6f0d-401f-bb88-3274657bf4ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.330029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681df6cb-6f0d-401f-bb88-3274657bf4ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:44.334639 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f48147-8f25-43d4-a163-53b5ca61b048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:44.335242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f48147-8f25-43d4-a163-53b5ca61b048 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.335515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f48147-8f25-43d4-a163-53b5ca61b048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.335780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f48147-8f25-43d4-a163-53b5ca61b048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.336419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f48147-8f25-43d4-a163-53b5ca61b048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:44.338054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f48147-8f25-43d4-a163-53b5ca61b048 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:44.348344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1103/2205] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:41:44.352406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3141026-5134-4645-99a6-6a22d838cd29 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.362490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3141026-5134-4645-99a6-6a22d838cd29 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:44.368011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2bc550-7923-4fb6-8303-640be5761006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:44.368547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc550-7923-4fb6-8303-640be5761006 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.368805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc550-7923-4fb6-8303-640be5761006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.369092 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2bc550-7923-4fb6-8303-640be5761006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.369813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bc550-7923-4fb6-8303-640be5761006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:44.371401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2bc550-7923-4fb6-8303-640be5761006 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:44.382104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1103/2206] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:41:44.386386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f48147-8f25-43d4-a163-53b5ca61b048 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.396271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f48147-8f25-43d4-a163-53b5ca61b048 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:44.401048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:44.401801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.402079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.402352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] PATH_INFO: `/v3/projects/2656e0af7bc649f9acca4a569e59d785/users/98ada3c8c98a485f910d1f5ecf383366/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.417740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2656e0af7bc649f9acca4a569e59d785, user_id=98ada3c8c98a485f910d1f5ecf383366, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:44.420381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:44.435154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1104/2207] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:41:44 2022] PUT /identity/v3/projects/2656e0af7bc649f9acca4a569e59d785/users/98ada3c8c98a485f910d1f5ecf383366/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:44.446582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333603b-f763-49a7-92e6-7c46fcf463e3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.446930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333603b-f763-49a7-92e6-7c46fcf463e3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.447301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333603b-f763-49a7-92e6-7c46fcf463e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.482750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f333603b-f763-49a7-92e6-7c46fcf463e3 None None] MFA Rules not processed for user `98ada3c8c98a485f910d1f5ecf383366`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:44.483943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f333603b-f763-49a7-92e6-7c46fcf463e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:44.550358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1104/2208] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:44.562384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.564402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:44.587412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:44.626871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fc0dc7-3fa0-4cc1-ad97-b9f25472fe37 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:44.631096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f370a3d-65c3-488d-8bbb-e80ca95ae145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:44.631704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f370a3d-65c3-488d-8bbb-e80ca95ae145 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.631979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f370a3d-65c3-488d-8bbb-e80ca95ae145 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.632248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f370a3d-65c3-488d-8bbb-e80ca95ae145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.640898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f370a3d-65c3-488d-8bbb-e80ca95ae145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:44.642507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f370a3d-65c3-488d-8bbb-e80ca95ae145 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:44.651434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1105/2209] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:44.683954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333603b-f763-49a7-92e6-7c46fcf463e3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.749059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333603b-f763-49a7-92e6-7c46fcf463e3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:44.754920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:44.755565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.755836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.756102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] PATH_INFO: `/v3/users/98ada3c8c98a485f910d1f5ecf383366` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.757249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ada3c8c98a485f910d1f5ecf383366)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:44.759267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:44.761871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:44.810821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ada3c8c98a485f910d1f5ecf383366'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:44.817626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ada3c8c98a485f910d1f5ecf383366'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:44.818313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ada3c8c98a485f910d1f5ecf383366'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:44.827162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ada3c8c98a485f910d1f5ecf383366'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:44.839057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ada3c8c98a485f910d1f5ecf383366'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:44.845353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1105/2210] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:41:44 2022] DELETE /identity/v3/users/98ada3c8c98a485f910d1f5ecf383366 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:44.849948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aa710-c6f3-4ec4-b27c-fa4f17a3a7c4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:44.850232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aa710-c6f3-4ec4-b27c-fa4f17a3a7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:44.850512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9aa710-c6f3-4ec4-b27c-fa4f17a3a7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:44.869466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ec9aa710-c6f3-4ec4-b27c-fa4f17a3a7c4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:44.870720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec9aa710-c6f3-4ec4-b27c-fa4f17a3a7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:44.934334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1106/2211] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:44.943075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:44.944617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.002232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1407fe-ecb0-4cb3-a86e-185e7f39d2b7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.005888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.006500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.006778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.007046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.015825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.017298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.026625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1106/2212] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:45.251465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9aa710-c6f3-4ec4-b27c-fa4f17a3a7c4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.253283 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ec9aa710-c6f3-4ec4-b27c-fa4f17a3a7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.265106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9aa710-c6f3-4ec4-b27c-fa4f17a3a7c4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.269605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.270268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.270543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.270810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] PATH_INFO: `/v3/users/a5650814bb1340fba9330c04d78fb347` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.285622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5650814bb1340fba9330c04d78fb347)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.287617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.302687 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.304182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.320515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f5bba-6361-4469-ba0d-a9093c57546b service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.328537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.329373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.329677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.329955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] PATH_INFO: `/v3/projects/2656e0af7bc649f9acca4a569e59d785` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.331176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2656e0af7bc649f9acca4a569e59d785)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.333179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.400224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2656e0af7bc649f9acca4a569e59d785', 'request_id': 'req-cad0bccf-2652-4d10-9b3b-2e05698468f3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.406376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2656e0af7bc649f9acca4a569e59d785', 'request_id': 'req-cad0bccf-2652-4d10-9b3b-2e05698468f3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.407384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5650814bb1340fba9330c04d78fb347'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.414142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2656e0af7bc649f9acca4a569e59d785', 'request_id': 'req-cad0bccf-2652-4d10-9b3b-2e05698468f3'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.416687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5650814bb1340fba9330c04d78fb347'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.418089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5650814bb1340fba9330c04d78fb347'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.426912 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5650814bb1340fba9330c04d78fb347'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.429094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] The token cache is being invalidate because project 2656e0af7bc649f9acca4a569e59d785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:41:45.429387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.430051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-cad0bccf-2652-4d10-9b3b-2e05698468f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.431543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1107/2213] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:41:45 2022] DELETE /identity/v3/projects/2656e0af7bc649f9acca4a569e59d785 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:45.436363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1cd8-ad32-4670-aeee-3f5a316dbe14 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.436645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1cd8-ad32-4670-aeee-3f5a316dbe14 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.436917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d1cd8-ad32-4670-aeee-3f5a316dbe14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.440017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-11fd56fa-ecdc-4619-abd7-a1a9fb65aed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5650814bb1340fba9330c04d78fb347'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:45.445876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1107/2214] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:41:45 2022] DELETE /identity/v3/users/a5650814bb1340fba9330c04d78fb347 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:45.450037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db07216-64fb-44bd-966c-dd03ed211f2a None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.450314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db07216-64fb-44bd-966c-dd03ed211f2a None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.450578 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db07216-64fb-44bd-966c-dd03ed211f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.454569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:45.460400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-729d1cd8-ad32-4670-aeee-3f5a316dbe14 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:45.462700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-729d1cd8-ad32-4670-aeee-3f5a316dbe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.472034 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9db07216-64fb-44bd-966c-dd03ed211f2a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:45.473497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9db07216-64fb-44bd-966c-dd03ed211f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.495092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:45.547353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1108/2215] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:41:45.550002 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1108/2216] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:41:45.551264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8b990-fe0c-4a4c-905a-fa2bbe085adc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.551517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8b990-fe0c-4a4c-905a-fa2bbe085adc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.551789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e8b990-fe0c-4a4c-905a-fa2bbe085adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.557651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d1cd8-ad32-4670-aeee-3f5a316dbe14 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.559085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-729d1cd8-ad32-4670-aeee-3f5a316dbe14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.574854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-69e8b990-fe0c-4a4c-905a-fa2bbe085adc None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:45.576393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-69e8b990-fe0c-4a4c-905a-fa2bbe085adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.585842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1109/2217] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:41:45.590464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8b990-fe0c-4a4c-905a-fa2bbe085adc None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.602572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e8b990-fe0c-4a4c-905a-fa2bbe085adc None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.608972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.609578 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.610064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.610334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.610978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:41:45.611272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.612851 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.621689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d1cd8-ad32-4670-aeee-3f5a316dbe14 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.625781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf1e48-d877-404d-bfb4-e545985f2537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.626460 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf1e48-d877-404d-bfb4-e545985f2537 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.629771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf1e48-d877-404d-bfb4-e545985f2537 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.629897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1110/2218] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:41:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:41:45.629983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf1e48-d877-404d-bfb4-e545985f2537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.631384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.649731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf1e48-d877-404d-bfb4-e545985f2537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.649731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf1e48-d877-404d-bfb4-e545985f2537 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.649891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8d45e6-7359-4666-bade-3fe917511e46 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.649891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32feea6-4d14-48e7-a1f1-82a7ce45f891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.649891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32feea6-4d14-48e7-a1f1-82a7ce45f891 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.649891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32feea6-4d14-48e7-a1f1-82a7ce45f891 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.650173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32feea6-4d14-48e7-a1f1-82a7ce45f891 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.650173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32feea6-4d14-48e7-a1f1-82a7ce45f891 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.650173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32feea6-4d14-48e7-a1f1-82a7ce45f891 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.650367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1109/2219] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:45.683235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1111/2220] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:41:45 2022] POST /identity/v3/projects => generated 391 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:41:45.684656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf1e48-d877-404d-bfb4-e545985f2537 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.694983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf1e48-d877-404d-bfb4-e545985f2537 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.751464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e1e76-371e-4245-bd62-7c14eabbc0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.752082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e1e76-371e-4245-bd62-7c14eabbc0fa admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.752358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e1e76-371e-4245-bd62-7c14eabbc0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.752627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0e1e76-371e-4245-bd62-7c14eabbc0fa admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.753289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e1e76-371e-4245-bd62-7c14eabbc0fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.755291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0e1e76-371e-4245-bd62-7c14eabbc0fa admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.782615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1110/2221] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:41:45 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:41:45.783914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32feea6-4d14-48e7-a1f1-82a7ce45f891 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.799399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32feea6-4d14-48e7-a1f1-82a7ce45f891 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.805259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cf0b6-0ce5-43cc-bd29-ae8635a713a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.805906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cf0b6-0ce5-43cc-bd29-ae8635a713a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.806180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cf0b6-0ce5-43cc-bd29-ae8635a713a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.806445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7cf0b6-0ce5-43cc-bd29-ae8635a713a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.807068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cf0b6-0ce5-43cc-bd29-ae8635a713a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.808739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7cf0b6-0ce5-43cc-bd29-ae8635a713a5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.818954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1112/2222] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:41:45.823166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e1e76-371e-4245-bd62-7c14eabbc0fa admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.836108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0e1e76-371e-4245-bd62-7c14eabbc0fa admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.841250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.842260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.842535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.842801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.843453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.847252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.861359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1111/2223] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:41:45.862587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cf0b6-0ce5-43cc-bd29-ae8635a713a5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.874953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7cf0b6-0ce5-43cc-bd29-ae8635a713a5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.883019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fdf34e-38f7-4d8e-bc16-7e11da456168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:45.883712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf34e-38f7-4d8e-bc16-7e11da456168 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.883990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf34e-38f7-4d8e-bc16-7e11da456168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.884260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fdf34e-38f7-4d8e-bc16-7e11da456168 admin admin] PATH_INFO: `/v3/projects/fe2a18b0ac7d4135828d5cec8cee04e7/users/8b6cdfa4ef794d028b9e0a012c0b729f/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.900197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdf34e-38f7-4d8e-bc16-7e11da456168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2a18b0ac7d4135828d5cec8cee04e7, user_id=8b6cdfa4ef794d028b9e0a012c0b729f, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:45.902974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fdf34e-38f7-4d8e-bc16-7e11da456168 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:45.930151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:45.930691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1113/2224] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:41:45 2022] PUT /identity/v3/projects/fe2a18b0ac7d4135828d5cec8cee04e7/users/8b6cdfa4ef794d028b9e0a012c0b729f/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:45.935605 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.940809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9c80e-a9d0-4881-8b70-d4cbe89c3589 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:45.941093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9c80e-a9d0-4881-8b70-d4cbe89c3589 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:45.941376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9c80e-a9d0-4881-8b70-d4cbe89c3589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:45.950441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106e1564-f18b-4be3-8362-69ea07076991 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:45.974716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f0b9c80e-a9d0-4881-8b70-d4cbe89c3589 None None] MFA Rules not processed for user `8b6cdfa4ef794d028b9e0a012c0b729f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:45.975956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f0b9c80e-a9d0-4881-8b70-d4cbe89c3589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:45.981097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:46.018842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:46.019487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:46.019760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:46.020028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] PATH_INFO: `/v3/projects/313d559efad04084bd8e6d7939fe4a72` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:46.031446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=313d559efad04084bd8e6d7939fe4a72)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:46.033427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:46.049562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1114/2225] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:41:46.060397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9c80e-a9d0-4881-8b70-d4cbe89c3589 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:46.061882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f0b9c80e-a9d0-4881-8b70-d4cbe89c3589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:46.086766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '313d559efad04084bd8e6d7939fe4a72', 'request_id': 'req-f2933fa3-b198-4148-a520-d4e160eb0856'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.093228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '313d559efad04084bd8e6d7939fe4a72', 'request_id': 'req-f2933fa3-b198-4148-a520-d4e160eb0856'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.104081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '313d559efad04084bd8e6d7939fe4a72', 'request_id': 'req-f2933fa3-b198-4148-a520-d4e160eb0856'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.106981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:46.119453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] The token cache is being invalidate because project 313d559efad04084bd8e6d7939fe4a72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:41:46.120004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.120639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.122272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1112/2226] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:41:45 2022] DELETE /identity/v3/projects/313d559efad04084bd8e6d7939fe4a72 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:41:46.123805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9c80e-a9d0-4881-8b70-d4cbe89c3589 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:46.126360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:46.127723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:46.172805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:46.173433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:46.173743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:46.174004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:46.176094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:46.191945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2933fa3-b198-4148-a520-d4e160eb0856 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:46.196932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:46.197676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:46.198028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:46.198363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] PATH_INFO: `/v3/users/414968e443074f93ae036da8ba465945` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:46.199594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=414968e443074f93ae036da8ba465945)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:46.201548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:46.242104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:46.244391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:46.254684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1115/2227] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:46 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:41:46.271023 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '414968e443074f93ae036da8ba465945'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.275386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:46.278407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '414968e443074f93ae036da8ba465945'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.280001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '414968e443074f93ae036da8ba465945'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.292115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '414968e443074f93ae036da8ba465945'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.304590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa243285-db1c-4da5-822a-cfbe1b985582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '414968e443074f93ae036da8ba465945'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.311005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1113/2228] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:41:46 2022] DELETE /identity/v3/users/414968e443074f93ae036da8ba465945 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:41:46.616203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:46.627659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ebaf18-1d7f-4441-b99b-a69fa014d826 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:46.686798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:46.687563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:46.687884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:46.699236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] PATH_INFO: `/v3/projects/89d1d241875c4b119b3ee9121c316bb6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:46.699236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d1d241875c4b119b3ee9121c316bb6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:46.699236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:46.738068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d1d241875c4b119b3ee9121c316bb6', 'request_id': 'req-d3c4c1d3-f495-47f6-a297-d6679537fe30'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.739125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:46.744425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d1d241875c4b119b3ee9121c316bb6', 'request_id': 'req-d3c4c1d3-f495-47f6-a297-d6679537fe30'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.751083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d1d241875c4b119b3ee9121c316bb6', 'request_id': 'req-d3c4c1d3-f495-47f6-a297-d6679537fe30'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.765372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] The token cache is being invalidate because project 89d1d241875c4b119b3ee9121c316bb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:41:46.765664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.766374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d3c4c1d3-f495-47f6-a297-d6679537fe30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:41:46.767896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1116/2229] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:41:46 2022] DELETE /identity/v3/projects/89d1d241875c4b119b3ee9121c316bb6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:46.773733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6373e65-1b85-493e-a7a5-39503115a029 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:46.774019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6373e65-1b85-493e-a7a5-39503115a029 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:46.774293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6373e65-1b85-493e-a7a5-39503115a029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:46.795228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b6373e65-1b85-493e-a7a5-39503115a029 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:46.798520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b6373e65-1b85-493e-a7a5-39503115a029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:46.872222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1114/2230] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:46.880297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6831dcb-824d-45a4-8317-e342ab4cf46e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:46.880583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6831dcb-824d-45a4-8317-e342ab4cf46e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:46.880855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6831dcb-824d-45a4-8317-e342ab4cf46e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:46.902861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a6831dcb-824d-45a4-8317-e342ab4cf46e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:46.904082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a6831dcb-824d-45a4-8317-e342ab4cf46e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:46.914015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1117/2231] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:46.918691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6373e65-1b85-493e-a7a5-39503115a029 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:46.930153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6373e65-1b85-493e-a7a5-39503115a029 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:46.935975 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:46.936620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:46.936898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:46.937157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:46.937843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:41:46.938156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:46.939918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:46.952727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1115/2232] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:41:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:41:46.957323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6831dcb-824d-45a4-8317-e342ab4cf46e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:46.966663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6831dcb-824d-45a4-8317-e342ab4cf46e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:46.971767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1cddd-b269-4b65-8670-3e2af7a84788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:46.972372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1cddd-b269-4b65-8670-3e2af7a84788 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:46.972670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1cddd-b269-4b65-8670-3e2af7a84788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:46.972935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1cddd-b269-4b65-8670-3e2af7a84788 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:46.973582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1cddd-b269-4b65-8670-3e2af7a84788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:46.975514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1cddd-b269-4b65-8670-3e2af7a84788 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.008425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1118/2233] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:41:46 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:41:47.012676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.022879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13c922-af25-41fb-8d0c-b31447f056a7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.088503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200dffb-c89e-4d8d-9ac8-3820b2879550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.089116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200dffb-c89e-4d8d-9ac8-3820b2879550 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.089380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200dffb-c89e-4d8d-9ac8-3820b2879550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.089670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200dffb-c89e-4d8d-9ac8-3820b2879550 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.090326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200dffb-c89e-4d8d-9ac8-3820b2879550 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.092641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4200dffb-c89e-4d8d-9ac8-3820b2879550 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.117784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1116/2234] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:41:47 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:41:47.123049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1cddd-b269-4b65-8670-3e2af7a84788 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.133918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1cddd-b269-4b65-8670-3e2af7a84788 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.138974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b3753-b63a-47c0-a9da-0398e0f2e280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.139585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b3753-b63a-47c0-a9da-0398e0f2e280 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.139871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b3753-b63a-47c0-a9da-0398e0f2e280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.140141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7b3753-b63a-47c0-a9da-0398e0f2e280 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.140795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b3753-b63a-47c0-a9da-0398e0f2e280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.142990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7b3753-b63a-47c0-a9da-0398e0f2e280 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.153987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1119/2235] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:41:47.158875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200dffb-c89e-4d8d-9ac8-3820b2879550 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.174502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200dffb-c89e-4d8d-9ac8-3820b2879550 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.180494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8026486-7963-41ac-8993-8137ee869c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.181106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026486-7963-41ac-8993-8137ee869c40 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.181384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026486-7963-41ac-8993-8137ee869c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.181650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8026486-7963-41ac-8993-8137ee869c40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.182304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8026486-7963-41ac-8993-8137ee869c40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.183894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8026486-7963-41ac-8993-8137ee869c40 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.194920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1117/2236] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:41:47.199133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b3753-b63a-47c0-a9da-0398e0f2e280 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.209780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7b3753-b63a-47c0-a9da-0398e0f2e280 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.215475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31998660-d81e-4830-bf16-48ded040d08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.216137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31998660-d81e-4830-bf16-48ded040d08c admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.216412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31998660-d81e-4830-bf16-48ded040d08c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.216675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31998660-d81e-4830-bf16-48ded040d08c admin admin] PATH_INFO: `/v3/projects/85643bc20d08435986d42268dfee3edd/users/449ebbb7e2564b91995baeafa5ca6862/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.232449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31998660-d81e-4830-bf16-48ded040d08c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85643bc20d08435986d42268dfee3edd, user_id=449ebbb7e2564b91995baeafa5ca6862, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.235264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31998660-d81e-4830-bf16-48ded040d08c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.250985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1120/2237] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:41:47 2022] PUT /identity/v3/projects/85643bc20d08435986d42268dfee3edd/users/449ebbb7e2564b91995baeafa5ca6862/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:47.262773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe525-1990-414a-9439-b3fe6bd08464 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.262773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe525-1990-414a-9439-b3fe6bd08464 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.263045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fe525-1990-414a-9439-b3fe6bd08464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.292513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-386fe525-1990-414a-9439-b3fe6bd08464 None None] MFA Rules not processed for user `449ebbb7e2564b91995baeafa5ca6862`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:47.293772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-386fe525-1990-414a-9439-b3fe6bd08464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:47.324526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:47.365554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1118/2238] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:47.370363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31998660-d81e-4830-bf16-48ded040d08c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.444203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31998660-d81e-4830-bf16-48ded040d08c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.450112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2012a2-7d33-4e54-88cf-dc1838c1ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.450722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2012a2-7d33-4e54-88cf-dc1838c1ae44 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.450992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2012a2-7d33-4e54-88cf-dc1838c1ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.451966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2012a2-7d33-4e54-88cf-dc1838c1ae44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.451966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2012a2-7d33-4e54-88cf-dc1838c1ae44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.453844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2012a2-7d33-4e54-88cf-dc1838c1ae44 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.487349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1121/2239] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:41:47 2022] POST /identity/v3/projects => generated 379 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:41:47.491514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fe525-1990-414a-9439-b3fe6bd08464 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.518159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fe525-1990-414a-9439-b3fe6bd08464 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.574655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ed443-ed2e-430c-9dcd-e168218dbadf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.575535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ed443-ed2e-430c-9dcd-e168218dbadf admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.575931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ed443-ed2e-430c-9dcd-e168218dbadf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.576295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ed443-ed2e-430c-9dcd-e168218dbadf admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.577215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ed443-ed2e-430c-9dcd-e168218dbadf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.579736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069ed443-ed2e-430c-9dcd-e168218dbadf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.606868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1119/2240] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:41:47 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:41:47.607919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2012a2-7d33-4e54-88cf-dc1838c1ae44 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.618684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2012a2-7d33-4e54-88cf-dc1838c1ae44 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.624302 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e4b25-6068-4ced-ad2e-833aa6119bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.624906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e4b25-6068-4ced-ad2e-833aa6119bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.625176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e4b25-6068-4ced-ad2e-833aa6119bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.625436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e4b25-6068-4ced-ad2e-833aa6119bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.626108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e4b25-6068-4ced-ad2e-833aa6119bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.627701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e4b25-6068-4ced-ad2e-833aa6119bf7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.641141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1122/2241] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:41:47.642901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ed443-ed2e-430c-9dcd-e168218dbadf admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.653347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069ed443-ed2e-430c-9dcd-e168218dbadf admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.659137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e55cd75-a5d4-4436-afca-4808959bb4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.659736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55cd75-a5d4-4436-afca-4808959bb4e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.660008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55cd75-a5d4-4436-afca-4808959bb4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.660269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e55cd75-a5d4-4436-afca-4808959bb4e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.660910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55cd75-a5d4-4436-afca-4808959bb4e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.662570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e55cd75-a5d4-4436-afca-4808959bb4e1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.672914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1120/2242] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:41:47.677513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e4b25-6068-4ced-ad2e-833aa6119bf7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.687536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e4b25-6068-4ced-ad2e-833aa6119bf7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.692517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424d49-0bfa-43e3-8a11-dce21bd2b8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.693179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424d49-0bfa-43e3-8a11-dce21bd2b8a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.693455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424d49-0bfa-43e3-8a11-dce21bd2b8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.693758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36424d49-0bfa-43e3-8a11-dce21bd2b8a0 admin admin] PATH_INFO: `/v3/projects/22f2f060f3ba46e3a12028f9f7a7758e/users/bf8580697ecd43c4848d92e4df9ea40b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.709182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36424d49-0bfa-43e3-8a11-dce21bd2b8a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f2f060f3ba46e3a12028f9f7a7758e, user_id=bf8580697ecd43c4848d92e4df9ea40b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.711905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36424d49-0bfa-43e3-8a11-dce21bd2b8a0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.727424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1123/2243] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:41:47 2022] PUT /identity/v3/projects/22f2f060f3ba46e3a12028f9f7a7758e/users/bf8580697ecd43c4848d92e4df9ea40b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:47.737442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d2f0a-0e1b-4b00-b9fe-b5796ca9f7ea None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.737762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d2f0a-0e1b-4b00-b9fe-b5796ca9f7ea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.738032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4d2f0a-0e1b-4b00-b9fe-b5796ca9f7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.766169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2a4d2f0a-0e1b-4b00-b9fe-b5796ca9f7ea None None] MFA Rules not processed for user `bf8580697ecd43c4848d92e4df9ea40b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:47.767090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2a4d2f0a-0e1b-4b00-b9fe-b5796ca9f7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:47.830208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1121/2244] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:47.834748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424d49-0bfa-43e3-8a11-dce21bd2b8a0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.861665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:47.897883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36424d49-0bfa-43e3-8a11-dce21bd2b8a0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.902731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533444c8-5ae1-42b1-80fe-cbd3afcc9e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:47.903335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533444c8-5ae1-42b1-80fe-cbd3afcc9e4f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:47.903606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533444c8-5ae1-42b1-80fe-cbd3afcc9e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:47.903847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533444c8-5ae1-42b1-80fe-cbd3afcc9e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:47.904484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533444c8-5ae1-42b1-80fe-cbd3afcc9e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:47.906442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533444c8-5ae1-42b1-80fe-cbd3afcc9e4f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:47.933777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1124/2245] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:41:47 2022] POST /identity/v3/projects => generated 379 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:41:47.937928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d2f0a-0e1b-4b00-b9fe-b5796ca9f7ea None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:47.949259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4d2f0a-0e1b-4b00-b9fe-b5796ca9f7ea None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:47.980939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:48.008315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fd214-f58a-493b-a916-7993ddc1db5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:48.009045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fd214-f58a-493b-a916-7993ddc1db5f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:48.009351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fd214-f58a-493b-a916-7993ddc1db5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:48.009593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fd214-f58a-493b-a916-7993ddc1db5f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:48.010261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fd214-f58a-493b-a916-7993ddc1db5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:48.012203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535fd214-f58a-493b-a916-7993ddc1db5f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:48.035851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1122/2246] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:41:47 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:41:48.040217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533444c8-5ae1-42b1-80fe-cbd3afcc9e4f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:48.050486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533444c8-5ae1-42b1-80fe-cbd3afcc9e4f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:48.055119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61beeb-112b-4152-8f38-6d51a59db831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:48.055715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61beeb-112b-4152-8f38-6d51a59db831 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:48.055988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61beeb-112b-4152-8f38-6d51a59db831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:48.056249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab61beeb-112b-4152-8f38-6d51a59db831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:48.056938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61beeb-112b-4152-8f38-6d51a59db831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:48.058474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab61beeb-112b-4152-8f38-6d51a59db831 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:48.069291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1125/2247] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:41:48.073491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fd214-f58a-493b-a916-7993ddc1db5f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:48.083987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535fd214-f58a-493b-a916-7993ddc1db5f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:48.089173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fc06a-f7d4-4638-acc2-2125e24d7a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:48.091232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fc06a-f7d4-4638-acc2-2125e24d7a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:48.091232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fc06a-f7d4-4638-acc2-2125e24d7a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:48.091232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fc06a-f7d4-4638-acc2-2125e24d7a51 admin admin] PATH_INFO: `/v3/projects/3724812353e6484f841138a8ff03207a/users/a64f85c9d1c14408a0dc3ddb2b3d2846/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:48.107957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fc06a-f7d4-4638-acc2-2125e24d7a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3724812353e6484f841138a8ff03207a, user_id=a64f85c9d1c14408a0dc3ddb2b3d2846, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:48.110829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264fc06a-f7d4-4638-acc2-2125e24d7a51 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:48.129674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1123/2248] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:41:48 2022] PUT /identity/v3/projects/3724812353e6484f841138a8ff03207a/users/a64f85c9d1c14408a0dc3ddb2b3d2846/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:48.133837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61beeb-112b-4152-8f38-6d51a59db831 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:48.144712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab61beeb-112b-4152-8f38-6d51a59db831 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:48.201702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eb430e-36e4-4cf1-93de-44ff638be956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:48.202319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb430e-36e4-4cf1-93de-44ff638be956 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:48.202593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb430e-36e4-4cf1-93de-44ff638be956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:48.202856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eb430e-36e4-4cf1-93de-44ff638be956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:48.203682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb430e-36e4-4cf1-93de-44ff638be956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:48.205339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eb430e-36e4-4cf1-93de-44ff638be956 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:48.215945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1126/2249] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:41:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:41:48.220547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fc06a-f7d4-4638-acc2-2125e24d7a51 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:48.231357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fc06a-f7d4-4638-acc2-2125e24d7a51 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:48.236717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:48.237378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:48.237675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:48.237945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] PATH_INFO: `/v3/projects/3724812353e6484f841138a8ff03207a/users/a64f85c9d1c14408a0dc3ddb2b3d2846/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:48.239947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3724812353e6484f841138a8ff03207a, user_id=a64f85c9d1c14408a0dc3ddb2b3d2846, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:48.242805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:48.254165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1124/2250] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:41:48 2022] PUT /identity/v3/projects/3724812353e6484f841138a8ff03207a/users/a64f85c9d1c14408a0dc3ddb2b3d2846/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:41:48.265532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c08ee-ee2c-4877-b9c3-b6ec9965974f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:48.265938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c08ee-ee2c-4877-b9c3-b6ec9965974f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:48.266327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c08ee-ee2c-4877-b9c3-b6ec9965974f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:48.295592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-470c08ee-ee2c-4877-b9c3-b6ec9965974f None None] MFA Rules not processed for user `a64f85c9d1c14408a0dc3ddb2b3d2846`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:41:48.296913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-470c08ee-ee2c-4877-b9c3-b6ec9965974f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:48.384986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1127/2251] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:41:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:41:48.396996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:48.398624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:48.455716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103e260-f0dc-4aee-a826-4155f742bbd0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:48.459343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c97487-fced-4aa3-93e7-7b67d21c2e66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:48.459967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c97487-fced-4aa3-93e7-7b67d21c2e66 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:48.460238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c97487-fced-4aa3-93e7-7b67d21c2e66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:48.460505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c97487-fced-4aa3-93e7-7b67d21c2e66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:48.470682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c97487-fced-4aa3-93e7-7b67d21c2e66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:48.472166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c97487-fced-4aa3-93e7-7b67d21c2e66 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:48.482014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1125/2252] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:48.528873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c08ee-ee2c-4877-b9c3-b6ec9965974f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:48.547148 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c08ee-ee2c-4877-b9c3-b6ec9965974f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:48.551279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:48.551897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:48.552449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:48.552718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:48.577943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:41:48.618889 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:48.620349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:48.629268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1128/2253] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:58.086791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c97487-fced-4aa3-93e7-7b67d21c2e66 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:41:58.096271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c97487-fced-4aa3-93e7-7b67d21c2e66 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:41:58.100711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:41:58.101316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:41:58.101588 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:41:58.101884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:41:58.103727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:41:58.169196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:41:58.170784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:41:58.180699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1126/2254] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:41:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:41:58.625385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=19 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:08.176181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=83,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:41.701130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:42:41.703337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:41.770895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231dc39-60b7-4d9d-8217-06a3faffd3ad service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:42:41.775083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:42:41.775658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:41.775925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:41.776184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:41.778254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:41.855270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:42:41.856902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:42:41.869580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1129/2255] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:42:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:42:43.050776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:42:43.052411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:43.065704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d607bd-5605-41ed-9e3c-3f707faa7beb service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:42:43.076421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22ae8c-1563-4cfa-8edd-d96543c11bdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:42:43.077051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ae8c-1563-4cfa-8edd-d96543c11bdb service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:43.077326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ae8c-1563-4cfa-8edd-d96543c11bdb service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:43.077595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ae8c-1563-4cfa-8edd-d96543c11bdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:43.079632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8e22ae8c-1563-4cfa-8edd-d96543c11bdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:43.094594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ae8c-1563-4cfa-8edd-d96543c11bdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:42:43.095956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ae8c-1563-4cfa-8edd-d96543c11bdb service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:42:43.112475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1127/2256] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:42:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:42:49.406140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:42:49.407515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:49.421659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd64772-ef49-4fdd-bdb1-dfa2ea71c47e service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:42:49.428683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:42:49.429197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:49.429516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:49.429829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:49.431956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:49.444982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:42:49.446668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:42:49.457545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1130/2257] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:42:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:42:53.107784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:54.752713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf014ba4-d50a-4172-bce7-3e97969a17de None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:54.753312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf014ba4-d50a-4172-bce7-3e97969a17de None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:54.753408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf014ba4-d50a-4172-bce7-3e97969a17de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:54.818772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-bf014ba4-d50a-4172-bce7-3e97969a17de None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:42:54.820333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bf014ba4-d50a-4172-bce7-3e97969a17de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:54.894547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1128/2258] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 14:42:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:42:54.938098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:42:54.939567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:55.038316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d448f-3408-44fc-a3fa-b75781e204e6 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:42:55.043839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8430efd-fdd8-4e5b-ae6f-47b97b394d3e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:42:55.044524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8430efd-fdd8-4e5b-ae6f-47b97b394d3e service swift] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:55.044805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8430efd-fdd8-4e5b-ae6f-47b97b394d3e service swift] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:55.045079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8430efd-fdd8-4e5b-ae6f-47b97b394d3e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:55.053318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8430efd-fdd8-4e5b-ae6f-47b97b394d3e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:42:55.054938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8430efd-fdd8-4e5b-ae6f-47b97b394d3e service swift] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:42:55.058385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:55.063584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1131/2259] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:42:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:42:55.574159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf014ba4-d50a-4172-bce7-3e97969a17de None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:42:55.584464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf014ba4-d50a-4172-bce7-3e97969a17de None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:42:55.588681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2b8a0-23f5-4675-bf76-42c8eb417d21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:42:55.589317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b8a0-23f5-4675-bf76-42c8eb417d21 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:55.589568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b8a0-23f5-4675-bf76-42c8eb417d21 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:55.589866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2b8a0-23f5-4675-bf76-42c8eb417d21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:55.592151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11d2b8a0-23f5-4675-bf76-42c8eb417d21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:55.688612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2b8a0-23f5-4675-bf76-42c8eb417d21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:42:55.690044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2b8a0-23f5-4675-bf76-42c8eb417d21 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:42:55.701142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1129/2260] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:42:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:42:59.038853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af577df-1495-48c7-b1ed-1d7e206a550e None None] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:59.039692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af577df-1495-48c7-b1ed-1d7e206a550e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:59.039968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af577df-1495-48c7-b1ed-1d7e206a550e None None] PATH_INFO: `/` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:59.041321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1132/2261] 10.222.0.23 () {54 vars in 921 bytes} [Wed Oct 5 14:42:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 05 14:42:59.072531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76354dd5-c686-4de8-ad1f-b952dd05fc21 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:59.073049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76354dd5-c686-4de8-ad1f-b952dd05fc21 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:59.073049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76354dd5-c686-4de8-ad1f-b952dd05fc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:59.125535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-76354dd5-c686-4de8-ad1f-b952dd05fc21 None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:42:59.127075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-76354dd5-c686-4de8-ad1f-b952dd05fc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:59.174644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:42:59.198796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1130/2262] 10.222.0.23 () {58 vars in 1018 bytes} [Wed Oct 5 14:42:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:42:59.212586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af577df-1495-48c7-b1ed-1d7e206a550e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:42:59.214328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4af577df-1495-48c7-b1ed-1d7e206a550e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:42:59.235058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af577df-1495-48c7-b1ed-1d7e206a550e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:42:59.241016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bd84e-f09e-414d-8e73-fecb9f15163e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:42:59.241787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bd84e-f09e-414d-8e73-fecb9f15163e service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:42:59.242061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bd84e-f09e-414d-8e73-fecb9f15163e service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:42:59.242510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7bd84e-f09e-414d-8e73-fecb9f15163e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:42:59.252453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bd84e-f09e-414d-8e73-fecb9f15163e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:42:59.254182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7bd84e-f09e-414d-8e73-fecb9f15163e service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:42:59.269737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1133/2263] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:42:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:02.650443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76354dd5-c686-4de8-ad1f-b952dd05fc21 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:02.651892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-76354dd5-c686-4de8-ad1f-b952dd05fc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:02.666026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76354dd5-c686-4de8-ad1f-b952dd05fc21 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:02.671553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:02.672152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:02.672428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:02.672699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] PATH_INFO: `/v3/users/f109f3d7076543d6bf16eac46a40b3e2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:02.673933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f109f3d7076543d6bf16eac46a40b3e2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:02.675894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:02.737059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f109f3d7076543d6bf16eac46a40b3e2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:02.744403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f109f3d7076543d6bf16eac46a40b3e2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:02.745111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f109f3d7076543d6bf16eac46a40b3e2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:02.753777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f109f3d7076543d6bf16eac46a40b3e2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:02.766074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f109f3d7076543d6bf16eac46a40b3e2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:02.772094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1131/2264] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:02 2022] DELETE /identity/v3/users/f109f3d7076543d6bf16eac46a40b3e2 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:02.955783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bd84e-f09e-414d-8e73-fecb9f15163e service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:02.969822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7bd84e-f09e-414d-8e73-fecb9f15163e service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:03.034159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:03.034803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:03.035072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:03.035554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] PATH_INFO: `/v3/projects/126541d90e9c43269898b1249aa6ee9b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:03.037553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=126541d90e9c43269898b1249aa6ee9b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:03.039903 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:03.108472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '126541d90e9c43269898b1249aa6ee9b', 'request_id': 'req-03734f40-77b2-41e0-b8eb-43b762515079'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.117791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '126541d90e9c43269898b1249aa6ee9b', 'request_id': 'req-03734f40-77b2-41e0-b8eb-43b762515079'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.130311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '126541d90e9c43269898b1249aa6ee9b', 'request_id': 'req-03734f40-77b2-41e0-b8eb-43b762515079'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.152154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] The token cache is being invalidate because project 126541d90e9c43269898b1249aa6ee9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:03.152447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.153167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.155064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1134/2265] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:02 2022] DELETE /identity/v3/projects/126541d90e9c43269898b1249aa6ee9b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:03.159172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:03.160736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:03.177775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbfd86-7474-4c5b-bab4-66003ddc14d0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:03.252843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:03.253507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:03.253821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:03.254097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] PATH_INFO: `/v3/users/e438d2ca78f54d329bbd3dce41a9ff65` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:03.268260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e438d2ca78f54d329bbd3dce41a9ff65)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:03.270253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:03.328620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:03.331478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e438d2ca78f54d329bbd3dce41a9ff65'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.339828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e438d2ca78f54d329bbd3dce41a9ff65'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.340567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e438d2ca78f54d329bbd3dce41a9ff65'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.350115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e438d2ca78f54d329bbd3dce41a9ff65'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.362466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3fda0a92-aa6c-4cfb-b423-e74b3f6c1b3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e438d2ca78f54d329bbd3dce41a9ff65'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.368592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1132/2266] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:03 2022] DELETE /identity/v3/users/e438d2ca78f54d329bbd3dce41a9ff65 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:03.556794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:03.558358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:03.571330 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03734f40-77b2-41e0-b8eb-43b762515079 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:03.632024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:03.632675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:03.632955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:03.633226 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] PATH_INFO: `/v3/projects/b188c61d7d1242459d7ca94e7b3438b4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:03.644412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b188c61d7d1242459d7ca94e7b3438b4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:03.646417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:03.654998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:03.699767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b188c61d7d1242459d7ca94e7b3438b4', 'request_id': 'req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.705670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b188c61d7d1242459d7ca94e7b3438b4', 'request_id': 'req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.712291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b188c61d7d1242459d7ca94e7b3438b4', 'request_id': 'req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.727273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] The token cache is being invalidate because project b188c61d7d1242459d7ca94e7b3438b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:03.727484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.728138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4567e9e2-fb0b-4b35-8a0b-574490ac4bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:03.729650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1135/2267] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:03 2022] DELETE /identity/v3/projects/b188c61d7d1242459d7ca94e7b3438b4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:03.734711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13943-1ac8-4350-9c3b-e74dec09cfbf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:03.735230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13943-1ac8-4350-9c3b-e74dec09cfbf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:03.735230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb13943-1ac8-4350-9c3b-e74dec09cfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:03.755153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7fb13943-1ac8-4350-9c3b-e74dec09cfbf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:03.756366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7fb13943-1ac8-4350-9c3b-e74dec09cfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:03.815651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1133/2268] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:03.825745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020282d-9185-4a21-9072-5378b44f05d7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:03.826061 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020282d-9185-4a21-9072-5378b44f05d7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:03.826335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020282d-9185-4a21-9072-5378b44f05d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:03.847141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a020282d-9185-4a21-9072-5378b44f05d7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:03.848343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a020282d-9185-4a21-9072-5378b44f05d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:03.856699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1136/2269] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:03.861055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb13943-1ac8-4350-9c3b-e74dec09cfbf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:03.872586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb13943-1ac8-4350-9c3b-e74dec09cfbf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:03.877536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:03.878177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:03.878459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:03.878723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:03.879374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:03.879683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:03.881285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:03.892967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1134/2270] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:43:03.897309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020282d-9185-4a21-9072-5378b44f05d7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:03.907006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020282d-9185-4a21-9072-5378b44f05d7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:03.913570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843c224-7384-49f4-bf0d-48e5ff8bbc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:03.914204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843c224-7384-49f4-bf0d-48e5ff8bbc17 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:03.914484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843c224-7384-49f4-bf0d-48e5ff8bbc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:03.914747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843c224-7384-49f4-bf0d-48e5ff8bbc17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:03.915386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843c224-7384-49f4-bf0d-48e5ff8bbc17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:03.917311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843c224-7384-49f4-bf0d-48e5ff8bbc17 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:03.958118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1137/2271] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:03 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:03.959379 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:03.970594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b0a3f-0b60-4956-91b2-b9542e93f728 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:04.029596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2dbc7-b4d3-43f4-8206-6147494cd9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:04.030231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2dbc7-b4d3-43f4-8206-6147494cd9da admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:04.030511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2dbc7-b4d3-43f4-8206-6147494cd9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:04.030771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2dbc7-b4d3-43f4-8206-6147494cd9da admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:04.031419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2dbc7-b4d3-43f4-8206-6147494cd9da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:04.033389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae2dbc7-b4d3-43f4-8206-6147494cd9da admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:04.055142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1135/2272] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:03 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:04.059219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843c224-7384-49f4-bf0d-48e5ff8bbc17 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:04.070293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843c224-7384-49f4-bf0d-48e5ff8bbc17 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:04.075965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877be6ad-b318-4946-9aa2-69571a9f90c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:04.076739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877be6ad-b318-4946-9aa2-69571a9f90c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:04.076984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877be6ad-b318-4946-9aa2-69571a9f90c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:04.077251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877be6ad-b318-4946-9aa2-69571a9f90c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:04.077934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877be6ad-b318-4946-9aa2-69571a9f90c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:04.079514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877be6ad-b318-4946-9aa2-69571a9f90c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:04.090266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1138/2273] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:04.095229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2dbc7-b4d3-43f4-8206-6147494cd9da admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:04.105425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2dbc7-b4d3-43f4-8206-6147494cd9da admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:04.110377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49619a1f-d580-40cf-af6e-df58e8d26c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:04.111015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49619a1f-d580-40cf-af6e-df58e8d26c99 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:04.111290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49619a1f-d580-40cf-af6e-df58e8d26c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:04.111552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49619a1f-d580-40cf-af6e-df58e8d26c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:04.112206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49619a1f-d580-40cf-af6e-df58e8d26c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:04.114130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49619a1f-d580-40cf-af6e-df58e8d26c99 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:04.125004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1136/2274] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:04.129851 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877be6ad-b318-4946-9aa2-69571a9f90c6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:04.141583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877be6ad-b318-4946-9aa2-69571a9f90c6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:04.151388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:04.152254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:04.152829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:04.153500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] PATH_INFO: `/v3/projects/31137b9776dd4dbf89dcc7b10ef4e65b/users/7859215cd50c4a4f84e27788457dfef6/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:04.169102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31137b9776dd4dbf89dcc7b10ef4e65b, user_id=7859215cd50c4a4f84e27788457dfef6, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:04.171926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:04.186150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1139/2275] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:04 2022] PUT /identity/v3/projects/31137b9776dd4dbf89dcc7b10ef4e65b/users/7859215cd50c4a4f84e27788457dfef6/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:04.191657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abd639-6cdd-4c48-ae85-49b7814776e8 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:04.191941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abd639-6cdd-4c48-ae85-49b7814776e8 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:04.192210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abd639-6cdd-4c48-ae85-49b7814776e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:04.212973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f5abd639-6cdd-4c48-ae85-49b7814776e8 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:04.214234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f5abd639-6cdd-4c48-ae85-49b7814776e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:04.234587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:04.277027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1137/2276] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:04.287924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:04.290619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:04.349657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d0627-2d0f-4f5f-a23c-84268adf573d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:04.353299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:04.353957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:04.354233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:04.354503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:04.363346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:04.364784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:04.374239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1140/2277] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:08.611452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6748c-127e-4933-ba75-fb4ce5c05cc7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:08.613889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6748c-127e-4933-ba75-fb4ce5c05cc7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:08.613889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6748c-127e-4933-ba75-fb4ce5c05cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:08.672339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c5f6748c-127e-4933-ba75-fb4ce5c05cc7 None None] MFA Rules not processed for user `7859215cd50c4a4f84e27788457dfef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:08.673728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c5f6748c-127e-4933-ba75-fb4ce5c05cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:08.763229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:08.765515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:08.782748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1138/2278] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:08.784433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6748c-127e-4933-ba75-fb4ce5c05cc7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:08.803830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6748c-127e-4933-ba75-fb4ce5c05cc7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:08.809446 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab61823-62eb-4860-893a-f660c98a7293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:08.810343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab61823-62eb-4860-893a-f660c98a7293 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:08.810474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab61823-62eb-4860-893a-f660c98a7293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:08.810726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab61823-62eb-4860-893a-f660c98a7293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:08.811408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab61823-62eb-4860-893a-f660c98a7293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:08.813504 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab61823-62eb-4860-893a-f660c98a7293 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:08.824529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:08.848290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077d082f-0ecb-4d3c-8815-8837ed13fd79 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:08.850695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1139/2279] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:08 2022] POST /identity/v3/projects => generated 373 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:08.859188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab61823-62eb-4860-893a-f660c98a7293 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:08.929454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:08.931792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:08.932251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:08.932714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:08.935763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:08.994293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab61823-62eb-4860-893a-f660c98a7293 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:09.001076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c7365-921c-41d1-aab7-0182de4715a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:09.002135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c7365-921c-41d1-aab7-0182de4715a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:09.003144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c7365-921c-41d1-aab7-0182de4715a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:09.003580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309c7365-921c-41d1-aab7-0182de4715a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:09.004573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c7365-921c-41d1-aab7-0182de4715a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:09.008268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309c7365-921c-41d1-aab7-0182de4715a7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:09.033992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:09.035677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:09.045720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1140/2280] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:08 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:09.046778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c7365-921c-41d1-aab7-0182de4715a7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:09.060064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309c7365-921c-41d1-aab7-0182de4715a7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:09.066391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1141/2281] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:09.069150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202e984-7113-4a70-a8f6-63b1fe160b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:09.069958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202e984-7113-4a70-a8f6-63b1fe160b34 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:09.070235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202e984-7113-4a70-a8f6-63b1fe160b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:09.070512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1202e984-7113-4a70-a8f6-63b1fe160b34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:09.071176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202e984-7113-4a70-a8f6-63b1fe160b34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:09.072799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1202e984-7113-4a70-a8f6-63b1fe160b34 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:09.084210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1141/2282] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:09.088232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:09.099603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f68b90-6d8b-4889-ad24-4b24852e036d service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:09.104310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19147419-6631-4d60-9d06-e29cf316e20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:09.104970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19147419-6631-4d60-9d06-e29cf316e20e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:09.105292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19147419-6631-4d60-9d06-e29cf316e20e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:09.105580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19147419-6631-4d60-9d06-e29cf316e20e admin admin] PATH_INFO: `/v3/projects/3a11431ac93b43948e378382b9018d9d/users/ceb2c96376ec456b83651cbe48a2590d/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:09.128271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19147419-6631-4d60-9d06-e29cf316e20e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a11431ac93b43948e378382b9018d9d, user_id=ceb2c96376ec456b83651cbe48a2590d, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:09.131291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19147419-6631-4d60-9d06-e29cf316e20e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:09.156436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1142/2283] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:09 2022] PUT /identity/v3/projects/3a11431ac93b43948e378382b9018d9d/users/ceb2c96376ec456b83651cbe48a2590d/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:09.158462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202e984-7113-4a70-a8f6-63b1fe160b34 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:09.182422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1202e984-7113-4a70-a8f6-63b1fe160b34 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:09.193923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:09.257140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:09.257914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:09.258408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:09.258806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:09.259499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:09.261157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:09.272712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1142/2284] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:09.277073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19147419-6631-4d60-9d06-e29cf316e20e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:09.288046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19147419-6631-4d60-9d06-e29cf316e20e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:09.294090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:09.294842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:09.295122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:09.295431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] PATH_INFO: `/v3/projects/3a11431ac93b43948e378382b9018d9d/users/ceb2c96376ec456b83651cbe48a2590d/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:09.298415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a11431ac93b43948e378382b9018d9d, user_id=ceb2c96376ec456b83651cbe48a2590d, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:09.304788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:09.320572 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1143/2285] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:09 2022] PUT /identity/v3/projects/3a11431ac93b43948e378382b9018d9d/users/ceb2c96376ec456b83651cbe48a2590d/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:12.771894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:12.773828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:12.901847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53889eef-fff0-4189-992b-1da472f40b7e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:13.015695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:13.016635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:13.016994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:13.017325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:13.019541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:13.148901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:13.150583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:13.172574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1143/2286] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:13.371137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:13.380311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:13.606691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:13.631156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af0a6b-f428-45c5-9249-9f4dcd9e92ac admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:13.644853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f574746-07a2-4f41-9d05-58a7db9bfc60 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417b661f770846f189c36d0804b6dce5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:13.647058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f574746-07a2-4f41-9d05-58a7db9bfc60 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:13.647344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f574746-07a2-4f41-9d05-58a7db9bfc60 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:13.647628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f574746-07a2-4f41-9d05-58a7db9bfc60 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:13.655076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5f574746-07a2-4f41-9d05-58a7db9bfc60 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-5f574746-07a2-4f41-9d05-58a7db9bfc60 service glance] Could not find project: 652d9098b8e5436fbc0542cd8f8cffbe.: keystone.exception.ProjectNotFound: Could not find project: 652d9098b8e5436fbc0542cd8f8cffbe. Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 05 14:43:13.672239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:43:13.673534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:43:13.674828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:43:13.676157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:43:13.677315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 652d9098b8e5436fbc0542cd8f8cffbe. Oct 05 14:43:13.677315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:43:13.679452 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1144/2287] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:13 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 05 14:43:14.668050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:14.670777 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:14.696064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:14.727823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d2a3-f72f-459b-877c-9c90b688ad8c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:14.728400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d2a3-f72f-459b-877c-9c90b688ad8c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:14.728511 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15d2a3-f72f-459b-877c-9c90b688ad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:14.767885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0a15d2a3-f72f-459b-877c-9c90b688ad8c None None] MFA Rules not processed for user `ceb2c96376ec456b83651cbe48a2590d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:14.769454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0a15d2a3-f72f-459b-877c-9c90b688ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:14.781102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f5c20-45bb-4ae4-a15e-3b5ce001d928 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:14.786119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:14.786774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:14.787068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:14.787332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] PATH_INFO: `/v3/users/80b5d2c96406427c81c2c425c8d1783b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:14.790394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=80b5d2c96406427c81c2c425c8d1783b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:14.793164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:14.884652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80b5d2c96406427c81c2c425c8d1783b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:14.890736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1145/2288] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:14.912251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80b5d2c96406427c81c2c425c8d1783b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:14.913052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80b5d2c96406427c81c2c425c8d1783b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:14.927861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15d2a3-f72f-459b-877c-9c90b688ad8c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:14.928207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80b5d2c96406427c81c2c425c8d1783b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:14.930435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0a15d2a3-f72f-459b-877c-9c90b688ad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:14.947317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-6ad4384e-e9b4-4105-92a1-d9a2a8240550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80b5d2c96406427c81c2c425c8d1783b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:14.968286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1144/2289] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:14 2022] DELETE /identity/v3/users/80b5d2c96406427c81c2c425c8d1783b => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:14.969937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113d00-a4dd-4abd-9ca1-a63a224c266b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:14.970226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113d00-a4dd-4abd-9ca1-a63a224c266b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:14.970501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c113d00-a4dd-4abd-9ca1-a63a224c266b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:15.007842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6c113d00-a4dd-4abd-9ca1-a63a224c266b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:15.009213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c113d00-a4dd-4abd-9ca1-a63a224c266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:15.089325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15d2a3-f72f-459b-877c-9c90b688ad8c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:15.090876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1145/2290] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:15.102408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113d00-a4dd-4abd-9ca1-a63a224c266b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:15.104099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6c113d00-a4dd-4abd-9ca1-a63a224c266b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:15.194897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:15.204349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96778d90-9ef9-428f-bb96-8072dfb1b888 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:15.205206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778d90-9ef9-428f-bb96-8072dfb1b888 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:15.205682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778d90-9ef9-428f-bb96-8072dfb1b888 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:15.206284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96778d90-9ef9-428f-bb96-8072dfb1b888 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:15.210739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-96778d90-9ef9-428f-bb96-8072dfb1b888 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:15.243032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c113d00-a4dd-4abd-9ca1-a63a224c266b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:15.246955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:15.247862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:15.248141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:15.248415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:15.254211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:15.282546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:15.285204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:15.301926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1146/2291] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:15.379246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96778d90-9ef9-428f-bb96-8072dfb1b888 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:15.381021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96778d90-9ef9-428f-bb96-8072dfb1b888 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:15.401505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1146/2292] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:15.547665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:15.550246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:15.584932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594e85e-57ca-4d77-b4c2-612802e6ceaf service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:15.590070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:15.591282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:15.591651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:15.592067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] PATH_INFO: `/v3/users/449ebbb7e2564b91995baeafa5ca6862` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:15.593574 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=449ebbb7e2564b91995baeafa5ca6862)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:15.595966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:15.668596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '449ebbb7e2564b91995baeafa5ca6862'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:15.677442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '449ebbb7e2564b91995baeafa5ca6862'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:15.682544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '449ebbb7e2564b91995baeafa5ca6862'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:15.698875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '449ebbb7e2564b91995baeafa5ca6862'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:15.723033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '449ebbb7e2564b91995baeafa5ca6862'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:15.731507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1147/2293] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:15 2022] DELETE /identity/v3/users/449ebbb7e2564b91995baeafa5ca6862 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:15.735855 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422336f9-20ed-4491-b42d-de3d2cb1cf8d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:15.736662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422336f9-20ed-4491-b42d-de3d2cb1cf8d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:15.737100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422336f9-20ed-4491-b42d-de3d2cb1cf8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:15.776574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-422336f9-20ed-4491-b42d-de3d2cb1cf8d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:15.778307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-422336f9-20ed-4491-b42d-de3d2cb1cf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:15.804657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:15.806270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:15.887022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1147/2294] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:15.896796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422336f9-20ed-4491-b42d-de3d2cb1cf8d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:15.898452 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-422336f9-20ed-4491-b42d-de3d2cb1cf8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:15.909273 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00537d8e-b74e-425d-a842-30a318465d50 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:15.930186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:15.931443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:15.932397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:15.932881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] PATH_INFO: `/v3/projects/014a5f65bff548008a4c3f51cb8bb3c9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:15.934861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=014a5f65bff548008a4c3f51cb8bb3c9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:15.937581 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:15.977538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422336f9-20ed-4491-b42d-de3d2cb1cf8d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:15.984328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2ee54-1fe2-41e7-9089-8ea3448c86ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:15.985570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2ee54-1fe2-41e7-9089-8ea3448c86ff service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:15.986133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2ee54-1fe2-41e7-9089-8ea3448c86ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:15.986665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2ee54-1fe2-41e7-9089-8ea3448c86ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:16.000135 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2ee54-1fe2-41e7-9089-8ea3448c86ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:16.001937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b2ee54-1fe2-41e7-9089-8ea3448c86ff service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:16.009900 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '014a5f65bff548008a4c3f51cb8bb3c9', 'request_id': 'req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.012326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:16.016974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1148/2295] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:16.020346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '014a5f65bff548008a4c3f51cb8bb3c9', 'request_id': 'req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.029613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '014a5f65bff548008a4c3f51cb8bb3c9', 'request_id': 'req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.044756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] The token cache is being invalidate because project 014a5f65bff548008a4c3f51cb8bb3c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:16.045127 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.045926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.048362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1148/2296] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:15 2022] DELETE /identity/v3/projects/014a5f65bff548008a4c3f51cb8bb3c9 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:16.052235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2ee54-1fe2-41e7-9089-8ea3448c86ff service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:16.062307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2ee54-1fe2-41e7-9089-8ea3448c86ff service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:16.122750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:16.131318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:16.132230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:16.132712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:16.133136 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] PATH_INFO: `/v3/users/48bf71f916ac469e8b9249feebc06530` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:16.134660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] RBAC: Authorizing `identity:delete_user(user_id=48bf71f916ac469e8b9249feebc06530)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:16.136691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:16.198759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48bf71f916ac469e8b9249feebc06530'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.209417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48bf71f916ac469e8b9249feebc06530'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.210317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48bf71f916ac469e8b9249feebc06530'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.220100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48bf71f916ac469e8b9249feebc06530'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.235389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48bf71f916ac469e8b9249feebc06530'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.242099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1149/2297] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:16 2022] DELETE /identity/v3/users/48bf71f916ac469e8b9249feebc06530 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:16.574034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:16.575527 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:16.598868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c515f4-c9d8-4171-99d5-6c24b5bb5a9a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:16.608931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:16.610844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:16.668664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:16.670312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:16.670791 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:16.671319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] PATH_INFO: `/v3/projects/85643bc20d08435986d42268dfee3edd` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:16.673620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] RBAC: Authorizing `identity:delete_project(project_id=85643bc20d08435986d42268dfee3edd)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:16.675188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:16.707732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce35ae0-50d6-46c2-abbe-4e087035127d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:16.713204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:16.713929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:16.714207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:16.714526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] PATH_INFO: `/v3/projects/aea0d39a4b4f45a19a6b9860f9525658` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:16.716202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aea0d39a4b4f45a19a6b9860f9525658)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:16.718446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:16.743914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85643bc20d08435986d42268dfee3edd', 'request_id': 'req-79a509a2-64be-486e-b167-1c835a82894e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.750710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85643bc20d08435986d42268dfee3edd', 'request_id': 'req-79a509a2-64be-486e-b167-1c835a82894e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.766479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85643bc20d08435986d42268dfee3edd', 'request_id': 'req-79a509a2-64be-486e-b167-1c835a82894e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.792468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] The token cache is being invalidate because project 85643bc20d08435986d42268dfee3edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:16.792750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.794255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.794926 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aea0d39a4b4f45a19a6b9860f9525658', 'request_id': 'req-1625c3e8-c374-4604-8c8d-8280e4deba95'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.797592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1149/2298] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:16 2022] DELETE /identity/v3/projects/85643bc20d08435986d42268dfee3edd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:16.803418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:16.803823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aea0d39a4b4f45a19a6b9860f9525658', 'request_id': 'req-1625c3e8-c374-4604-8c8d-8280e4deba95'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.807868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:16.813495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aea0d39a4b4f45a19a6b9860f9525658', 'request_id': 'req-1625c3e8-c374-4604-8c8d-8280e4deba95'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.840444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] The token cache is being invalidate because project aea0d39a4b4f45a19a6b9860f9525658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:16.841896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.843378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-1625c3e8-c374-4604-8c8d-8280e4deba95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:16.848497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1150/2299] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:16 2022] DELETE /identity/v3/projects/aea0d39a4b4f45a19a6b9860f9525658 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:16.855550 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d11280-ce1f-4fe9-aedb-b972574fd77c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:16.856821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d11280-ce1f-4fe9-aedb-b972574fd77c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:16.858141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d11280-ce1f-4fe9-aedb-b972574fd77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:16.880470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:16.909722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-00d11280-ce1f-4fe9-aedb-b972574fd77c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:16.911097 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-00d11280-ce1f-4fe9-aedb-b972574fd77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:16.930163 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a509a2-64be-486e-b167-1c835a82894e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:16.946268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:17.011164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1151/2300] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:17.013287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682936ec-3c3c-4583-8d8b-f4efbeb338be None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.014144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682936ec-3c3c-4583-8d8b-f4efbeb338be None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.014144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682936ec-3c3c-4583-8d8b-f4efbeb338be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.019498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.020153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.020617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.020856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] PATH_INFO: `/v3/users/bf8580697ecd43c4848d92e4df9ea40b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.022060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf8580697ecd43c4848d92e4df9ea40b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.024242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.057603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-682936ec-3c3c-4583-8d8b-f4efbeb338be None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:17.059134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-682936ec-3c3c-4583-8d8b-f4efbeb338be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:17.073033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1152/2301] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:17.077713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed84e3f-c89c-4a46-9007-d29ab7fd80d4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.078071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed84e3f-c89c-4a46-9007-d29ab7fd80d4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.078418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed84e3f-c89c-4a46-9007-d29ab7fd80d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.089282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf8580697ecd43c4848d92e4df9ea40b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.098109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf8580697ecd43c4848d92e4df9ea40b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.098901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf8580697ecd43c4848d92e4df9ea40b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.108502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3ed84e3f-c89c-4a46-9007-d29ab7fd80d4 None None] MFA Rules not processed for user `7859215cd50c4a4f84e27788457dfef6`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:17.109845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3ed84e3f-c89c-4a46-9007-d29ab7fd80d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:17.110793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf8580697ecd43c4848d92e4df9ea40b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.127806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf8580697ecd43c4848d92e4df9ea40b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.137442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1150/2302] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:16 2022] DELETE /identity/v3/users/bf8580697ecd43c4848d92e4df9ea40b => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:17.137442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.137442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:17.185388 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1153/2303] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:17.260980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb03730-85b6-42cf-ba2d-00e1dfaee190 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:17.266994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.267796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.268249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.268664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.269476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:17.269978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.271726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.291922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1151/2304] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:43:17.292996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed84e3f-c89c-4a46-9007-d29ab7fd80d4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.310056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed84e3f-c89c-4a46-9007-d29ab7fd80d4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:17.316484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a71f3-32f7-43b9-841d-e3f3d1eecbec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.317229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a71f3-32f7-43b9-841d-e3f3d1eecbec admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.317671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a71f3-32f7-43b9-841d-e3f3d1eecbec admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.317808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a71f3-32f7-43b9-841d-e3f3d1eecbec admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.318448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a71f3-32f7-43b9-841d-e3f3d1eecbec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.320508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a71f3-32f7-43b9-841d-e3f3d1eecbec admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.362776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1154/2305] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:17 2022] POST /identity/v3/projects => generated 365 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:17.366684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.380656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79936d-b2fd-4f22-8a29-6a69168868a1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:17.434418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:17.478842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.479539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.479884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.480550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.480967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.483101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.517532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1152/2306] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:17 2022] POST /identity/v3/users => generated 283 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:17.521492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a71f3-32f7-43b9-841d-e3f3d1eecbec admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.537898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.540915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:17.544434 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a71f3-32f7-43b9-841d-e3f3d1eecbec admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:17.560677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0dde7-3bcd-465f-a398-ff553f2c1eda admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:17.560941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27408f-7284-4783-bdb4-9a2042019902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.562373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27408f-7284-4783-bdb4-9a2042019902 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.563257 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27408f-7284-4783-bdb4-9a2042019902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.563555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27408f-7284-4783-bdb4-9a2042019902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.564277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27408f-7284-4783-bdb4-9a2042019902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.566148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.566633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27408f-7284-4783-bdb4-9a2042019902 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.566934 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.567382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.567729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] PATH_INFO: `/v3/projects/22f2f060f3ba46e3a12028f9f7a7758e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.569302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f2f060f3ba46e3a12028f9f7a7758e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.571969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.584193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1155/2307] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:17.585426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27408f-7284-4783-bdb4-9a2042019902 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.603504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27408f-7284-4783-bdb4-9a2042019902 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:17.614016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27690b29-d6d9-4f55-b99a-9687848a21ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.615128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27690b29-d6d9-4f55-b99a-9687848a21ba admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.615558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27690b29-d6d9-4f55-b99a-9687848a21ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.615837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27690b29-d6d9-4f55-b99a-9687848a21ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.616710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27690b29-d6d9-4f55-b99a-9687848a21ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.618408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27690b29-d6d9-4f55-b99a-9687848a21ba admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.631053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1156/2308] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:17 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:17.635319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27690b29-d6d9-4f55-b99a-9687848a21ba admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.648125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27690b29-d6d9-4f55-b99a-9687848a21ba admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:17.653576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f2f060f3ba46e3a12028f9f7a7758e', 'request_id': 'req-c72d43d3-5bce-480c-975f-48edd20eb09d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.667257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f2f060f3ba46e3a12028f9f7a7758e', 'request_id': 'req-c72d43d3-5bce-480c-975f-48edd20eb09d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.680341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f2f060f3ba46e3a12028f9f7a7758e', 'request_id': 'req-c72d43d3-5bce-480c-975f-48edd20eb09d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.694853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] The token cache is being invalidate because project 22f2f060f3ba46e3a12028f9f7a7758e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:17.695048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.695830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:17.697996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1153/2309] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:17 2022] DELETE /identity/v3/projects/22f2f060f3ba46e3a12028f9f7a7758e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:17.701472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.704145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:17.743288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7545105-05f4-4c47-b824-8cc2aa95a877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.744328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7545105-05f4-4c47-b824-8cc2aa95a877 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.744768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7545105-05f4-4c47-b824-8cc2aa95a877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.744965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7545105-05f4-4c47-b824-8cc2aa95a877 admin admin] PATH_INFO: `/v3/projects/f93322b3b1c64fd3b2037d4ee2af8202/users/3e945d84eaae4953afc5e39604d1974b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.748635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:17.767167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7545105-05f4-4c47-b824-8cc2aa95a877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93322b3b1c64fd3b2037d4ee2af8202, user_id=3e945d84eaae4953afc5e39604d1974b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.770709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7545105-05f4-4c47-b824-8cc2aa95a877 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.801820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1157/2310] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:17 2022] PUT /identity/v3/projects/f93322b3b1c64fd3b2037d4ee2af8202/users/3e945d84eaae4953afc5e39604d1974b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:17.813656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2589aaa-e929-4986-a49a-fa91f135ab07 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.814150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2589aaa-e929-4986-a49a-fa91f135ab07 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.814597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2589aaa-e929-4986-a49a-fa91f135ab07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.833449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d43d3-5bce-480c-975f-48edd20eb09d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:17.881149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d2589aaa-e929-4986-a49a-fa91f135ab07 None None] MFA Rules not processed for user `3e945d84eaae4953afc5e39604d1974b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:17.882608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d2589aaa-e929-4986-a49a-fa91f135ab07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:17.910842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:17.927938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:17.929122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:17.929398 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:17.929695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] PATH_INFO: `/v3/users/a64f85c9d1c14408a0dc3ddb2b3d2846` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:17.931841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64f85c9d1c14408a0dc3ddb2b3d2846)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:17.934379 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:17.994703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1158/2311] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:17.996144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2589aaa-e929-4986-a49a-fa91f135ab07 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:17.998781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d2589aaa-e929-4986-a49a-fa91f135ab07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:18.017180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2589aaa-e929-4986-a49a-fa91f135ab07 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.022083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.022908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.023689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.023689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.024278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.027007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.027850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64f85c9d1c14408a0dc3ddb2b3d2846'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.040324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64f85c9d1c14408a0dc3ddb2b3d2846'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.041296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64f85c9d1c14408a0dc3ddb2b3d2846'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.052466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64f85c9d1c14408a0dc3ddb2b3d2846'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.064444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1159/2312] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:17 2022] POST /identity/v3/projects => generated 365 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:18.069602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.070043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64f85c9d1c14408a0dc3ddb2b3d2846'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.072197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:18.085147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1154/2313] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:17 2022] DELETE /identity/v3/users/a64f85c9d1c14408a0dc3ddb2b3d2846 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:18.157916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd577c96-d79f-4719-af9d-33107a712eef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.222073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:18.228674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ee7c5-1689-4557-8832-79c8e31af094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.229305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ee7c5-1689-4557-8832-79c8e31af094 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.229766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ee7c5-1689-4557-8832-79c8e31af094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.230054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ee7c5-1689-4557-8832-79c8e31af094 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.230848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ee7c5-1689-4557-8832-79c8e31af094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.233185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ee7c5-1689-4557-8832-79c8e31af094 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.273580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1160/2314] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:18 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:18.276307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.287368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd552-9c56-41bf-ab88-3cfae4692904 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.292282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a27e13-6071-4834-83c0-cb3f258d0a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.293107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a27e13-6071-4834-83c0-cb3f258d0a58 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.293440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a27e13-6071-4834-83c0-cb3f258d0a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.293808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a27e13-6071-4834-83c0-cb3f258d0a58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.294763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a27e13-6071-4834-83c0-cb3f258d0a58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.296440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a27e13-6071-4834-83c0-cb3f258d0a58 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.309089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1155/2315] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:18.313398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ee7c5-1689-4557-8832-79c8e31af094 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.324600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ee7c5-1689-4557-8832-79c8e31af094 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.330592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.331312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.331596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.331858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] PATH_INFO: `/v3/projects/c0cfa24b5b7f47aab13b4bf795f65881/users/930376f7040447418b2573dbd27b9dbb/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.369507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0cfa24b5b7f47aab13b4bf795f65881, user_id=930376f7040447418b2573dbd27b9dbb, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.372366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.391606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1161/2316] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:18 2022] PUT /identity/v3/projects/c0cfa24b5b7f47aab13b4bf795f65881/users/930376f7040447418b2573dbd27b9dbb/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:18.395993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a27e13-6071-4834-83c0-cb3f258d0a58 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.410783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.415204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a27e13-6071-4834-83c0-cb3f258d0a58 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.416925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:18.505884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285abc7-9e26-4433-a5db-455d370b3dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.506729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285abc7-9e26-4433-a5db-455d370b3dfc admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.507072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285abc7-9e26-4433-a5db-455d370b3dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.507407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285abc7-9e26-4433-a5db-455d370b3dfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.508137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285abc7-9e26-4433-a5db-455d370b3dfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.510064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285abc7-9e26-4433-a5db-455d370b3dfc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.512117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238cf85-a5d2-4677-aedd-9bd00b2c6cc7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.517086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.517858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.518198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.518529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] PATH_INFO: `/v3/projects/3724812353e6484f841138a8ff03207a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.519771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3724812353e6484f841138a8ff03207a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.521821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.525693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1156/2317] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:18.527662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285abc7-9e26-4433-a5db-455d370b3dfc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.539332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285abc7-9e26-4433-a5db-455d370b3dfc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.544231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61153a63-201e-4c4e-847d-a397ade62fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.544922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153a63-201e-4c4e-847d-a397ade62fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.545200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153a63-201e-4c4e-847d-a397ade62fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.545528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61153a63-201e-4c4e-847d-a397ade62fd5 admin admin] PATH_INFO: `/v3/projects/c0cfa24b5b7f47aab13b4bf795f65881/users/930376f7040447418b2573dbd27b9dbb/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.547988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61153a63-201e-4c4e-847d-a397ade62fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0cfa24b5b7f47aab13b4bf795f65881, user_id=930376f7040447418b2573dbd27b9dbb, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.550956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61153a63-201e-4c4e-847d-a397ade62fd5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.569707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1157/2318] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:18 2022] PUT /identity/v3/projects/c0cfa24b5b7f47aab13b4bf795f65881/users/930376f7040447418b2573dbd27b9dbb/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:18.574067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3724812353e6484f841138a8ff03207a', 'request_id': 'req-6d04a560-b9b8-4815-bc38-8c311a81a861'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.575617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f247-d5fd-4193-8ab5-f941fc27c866 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.575955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f247-d5fd-4193-8ab5-f941fc27c866 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.576591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a2f247-d5fd-4193-8ab5-f941fc27c866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.580373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3724812353e6484f841138a8ff03207a', 'request_id': 'req-6d04a560-b9b8-4815-bc38-8c311a81a861'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.589186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3724812353e6484f841138a8ff03207a', 'request_id': 'req-6d04a560-b9b8-4815-bc38-8c311a81a861'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.607754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] The token cache is being invalidate because project 3724812353e6484f841138a8ff03207a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:18.607966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.608692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6d04a560-b9b8-4815-bc38-8c311a81a861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:18.611231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1162/2319] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:18 2022] DELETE /identity/v3/projects/3724812353e6484f841138a8ff03207a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:18.612427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-01a2f247-d5fd-4193-8ab5-f941fc27c866 None None] MFA Rules not processed for user `930376f7040447418b2573dbd27b9dbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:18.614365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01a2f247-d5fd-4193-8ab5-f941fc27c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:18.616354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533bcf3-2521-455a-a0e9-95bb28ed2517 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.616636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533bcf3-2521-455a-a0e9-95bb28ed2517 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.617069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533bcf3-2521-455a-a0e9-95bb28ed2517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.643448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8533bcf3-2521-455a-a0e9-95bb28ed2517 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:18.644692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8533bcf3-2521-455a-a0e9-95bb28ed2517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:18.652878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:18.698366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:18.701367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1158/2320] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:18.713521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2f247-d5fd-4193-8ab5-f941fc27c866 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.715073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-01a2f247-d5fd-4193-8ab5-f941fc27c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:18.717240 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1163/2321] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:18.725591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92f1f6-e133-4cf6-bc5b-a9b334b82218 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.725902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92f1f6-e133-4cf6-bc5b-a9b334b82218 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.726173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92f1f6-e133-4cf6-bc5b-a9b334b82218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.746877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-af92f1f6-e133-4cf6-bc5b-a9b334b82218 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:18.748125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-af92f1f6-e133-4cf6-bc5b-a9b334b82218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:18.757195 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1164/2322] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:18.761102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92f1f6-e133-4cf6-bc5b-a9b334b82218 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.770515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92f1f6-e133-4cf6-bc5b-a9b334b82218 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.775501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.776122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.776396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.776663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.777318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:18.777614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.779315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.780851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a2f247-d5fd-4193-8ab5-f941fc27c866 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.784541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42695f35-d641-4f6d-adfb-522d84512c29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.785219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42695f35-d641-4f6d-adfb-522d84512c29 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.785562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42695f35-d641-4f6d-adfb-522d84512c29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.785913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42695f35-d641-4f6d-adfb-522d84512c29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.792074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1165/2323] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:43:18.796281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.796554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42695f35-d641-4f6d-adfb-522d84512c29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.798006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42695f35-d641-4f6d-adfb-522d84512c29 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.807815 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d4bc5-ca9c-4aa2-9338-3b8470bdc5f7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.808322 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1159/2324] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:18 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:18.815026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af8aba-a3e8-4c22-a19e-e28fe4f86744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.815675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af8aba-a3e8-4c22-a19e-e28fe4f86744 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.815959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af8aba-a3e8-4c22-a19e-e28fe4f86744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.816267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af8aba-a3e8-4c22-a19e-e28fe4f86744 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.816886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af8aba-a3e8-4c22-a19e-e28fe4f86744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.818831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af8aba-a3e8-4c22-a19e-e28fe4f86744 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.856191 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1166/2325] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:18 2022] POST /identity/v3/projects => generated 373 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:18.860389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42695f35-d641-4f6d-adfb-522d84512c29 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.871448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42695f35-d641-4f6d-adfb-522d84512c29 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:18.941471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e19e8-13e4-4b91-8a3e-f614aef1c37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:18.942136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e19e8-13e4-4b91-8a3e-f614aef1c37a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:18.942393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e19e8-13e4-4b91-8a3e-f614aef1c37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:18.942657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e19e8-13e4-4b91-8a3e-f614aef1c37a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:18.943300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e19e8-13e4-4b91-8a3e-f614aef1c37a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:18.945226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e19e8-13e4-4b91-8a3e-f614aef1c37a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:18.974668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1160/2326] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:18 2022] POST /identity/v3/users => generated 287 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:18.979897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af8aba-a3e8-4c22-a19e-e28fe4f86744 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:18.995546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af8aba-a3e8-4c22-a19e-e28fe4f86744 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:19.000391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f733c97-6d5f-4bc5-9d53-4e0e664a8f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:19.001017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f733c97-6d5f-4bc5-9d53-4e0e664a8f3a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.001291 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f733c97-6d5f-4bc5-9d53-4e0e664a8f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.001549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f733c97-6d5f-4bc5-9d53-4e0e664a8f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.002241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f733c97-6d5f-4bc5-9d53-4e0e664a8f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:19.003847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f733c97-6d5f-4bc5-9d53-4e0e664a8f3a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:19.015651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1167/2327] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:19.019580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e19e8-13e4-4b91-8a3e-f614aef1c37a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:19.032074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e19e8-13e4-4b91-8a3e-f614aef1c37a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:19.036844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0852ba9-bcd8-4a6d-a772-e2a569d6a48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:19.037546 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0852ba9-bcd8-4a6d-a772-e2a569d6a48f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.037910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0852ba9-bcd8-4a6d-a772-e2a569d6a48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.038239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0852ba9-bcd8-4a6d-a772-e2a569d6a48f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.039084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0852ba9-bcd8-4a6d-a772-e2a569d6a48f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:19.040772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0852ba9-bcd8-4a6d-a772-e2a569d6a48f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:19.055783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1161/2328] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:19.055887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f733c97-6d5f-4bc5-9d53-4e0e664a8f3a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:19.067501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f733c97-6d5f-4bc5-9d53-4e0e664a8f3a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:19.072385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858b71a-6ef4-4626-9632-4df3701f1803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:19.073115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858b71a-6ef4-4626-9632-4df3701f1803 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.073480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858b71a-6ef4-4626-9632-4df3701f1803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.073837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858b71a-6ef4-4626-9632-4df3701f1803 admin admin] PATH_INFO: `/v3/projects/41bde39f47f043acba3c006222877b68/users/c66e2822f7bc4abc8eb7d50e0a7fea6a/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.105684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858b71a-6ef4-4626-9632-4df3701f1803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41bde39f47f043acba3c006222877b68, user_id=c66e2822f7bc4abc8eb7d50e0a7fea6a, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:19.110432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858b71a-6ef4-4626-9632-4df3701f1803 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:19.132218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1168/2329] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:19 2022] PUT /identity/v3/projects/41bde39f47f043acba3c006222877b68/users/c66e2822f7bc4abc8eb7d50e0a7fea6a/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:19.144376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31cd1-419f-4a12-a398-6533d1796e88 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.144659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31cd1-419f-4a12-a398-6533d1796e88 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.144930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d31cd1-419f-4a12-a398-6533d1796e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.173952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-80d31cd1-419f-4a12-a398-6533d1796e88 None None] MFA Rules not processed for user `c66e2822f7bc4abc8eb7d50e0a7fea6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:19.174371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-80d31cd1-419f-4a12-a398-6533d1796e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:19.240990 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:19.286073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1162/2330] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:19.300557 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858b71a-6ef4-4626-9632-4df3701f1803 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:19.377881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858b71a-6ef4-4626-9632-4df3701f1803 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:19.382153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a310a6-8b3b-437a-b6e0-2120caf0b964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:19.382797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a310a6-8b3b-437a-b6e0-2120caf0b964 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.383074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a310a6-8b3b-437a-b6e0-2120caf0b964 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.383342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a310a6-8b3b-437a-b6e0-2120caf0b964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.393535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a310a6-8b3b-437a-b6e0-2120caf0b964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:19.395083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a310a6-8b3b-437a-b6e0-2120caf0b964 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:19.404563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1169/2331] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:19.428021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d31cd1-419f-4a12-a398-6533d1796e88 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:19.525016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d31cd1-419f-4a12-a398-6533d1796e88 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:19.525016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:19.525016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.525016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.525412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] PATH_INFO: `/v3/users/c66e2822f7bc4abc8eb7d50e0a7fea6a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.526482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c66e2822f7bc4abc8eb7d50e0a7fea6a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:19.528405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:19.584206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c66e2822f7bc4abc8eb7d50e0a7fea6a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.594032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c66e2822f7bc4abc8eb7d50e0a7fea6a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.594771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c66e2822f7bc4abc8eb7d50e0a7fea6a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.604400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c66e2822f7bc4abc8eb7d50e0a7fea6a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.618394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c66e2822f7bc4abc8eb7d50e0a7fea6a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.625364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1163/2332] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:19 2022] DELETE /identity/v3/users/c66e2822f7bc4abc8eb7d50e0a7fea6a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:19.630088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291166-f359-4a3f-bd17-fa9a3b91001b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.630364 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291166-f359-4a3f-bd17-fa9a3b91001b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.630686 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1291166-f359-4a3f-bd17-fa9a3b91001b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.657374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:19.658000 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a1291166-f359-4a3f-bd17-fa9a3b91001b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:19.659074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:19.659599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a1291166-f359-4a3f-bd17-fa9a3b91001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:19.722703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1170/2333] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:19.732718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7de6db-3d4a-47c2-996a-5886a66c1965 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:19.733961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:19.735374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1291166-f359-4a3f-bd17-fa9a3b91001b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:19.737489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a1291166-f359-4a3f-bd17-fa9a3b91001b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:19.739933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:19.740715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.741214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.741742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] PATH_INFO: `/v3/users/3e945d84eaae4953afc5e39604d1974b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.743184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e945d84eaae4953afc5e39604d1974b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:19.746095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:19.762371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:19.799399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1291166-f359-4a3f-bd17-fa9a3b91001b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:19.800672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e945d84eaae4953afc5e39604d1974b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.803248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04ba7d-25d0-4934-8fb8-09e0dc63751b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:19.803878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04ba7d-25d0-4934-8fb8-09e0dc63751b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.804150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04ba7d-25d0-4934-8fb8-09e0dc63751b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.804413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04ba7d-25d0-4934-8fb8-09e0dc63751b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.807384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e945d84eaae4953afc5e39604d1974b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.808047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e945d84eaae4953afc5e39604d1974b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.813483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04ba7d-25d0-4934-8fb8-09e0dc63751b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:19.815076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b04ba7d-25d0-4934-8fb8-09e0dc63751b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:19.817603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e945d84eaae4953afc5e39604d1974b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.824736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1171/2334] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:19.830184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e945d84eaae4953afc5e39604d1974b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:19.836361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1164/2335] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:19 2022] DELETE /identity/v3/users/3e945d84eaae4953afc5e39604d1974b => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:19.841275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a7340-c8e8-4521-8567-c53aa49cba38 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:19.841564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a7340-c8e8-4521-8567-c53aa49cba38 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:19.841842 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a7340-c8e8-4521-8567-c53aa49cba38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:19.862505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a70a7340-c8e8-4521-8567-c53aa49cba38 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:19.863823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a70a7340-c8e8-4521-8567-c53aa49cba38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:19.937783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1172/2336] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:19.947493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:19.949139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.030433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96dd52b-aec1-49de-a641-3d0df0168881 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.035349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bc413-8823-40d6-9994-ce8393e937e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.036106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bc413-8823-40d6-9994-ce8393e937e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.036386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bc413-8823-40d6-9994-ce8393e937e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.036655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bc413-8823-40d6-9994-ce8393e937e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.046420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bc413-8823-40d6-9994-ce8393e937e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.047950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bc413-8823-40d6-9994-ce8393e937e0 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.058868 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1165/2337] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:20.136904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a7340-c8e8-4521-8567-c53aa49cba38 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.139105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a70a7340-c8e8-4521-8567-c53aa49cba38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.159798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a7340-c8e8-4521-8567-c53aa49cba38 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.174821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.176030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.176326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.177140 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] PATH_INFO: `/v3/projects/41bde39f47f043acba3c006222877b68` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.179688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] RBAC: Authorizing `identity:delete_project(project_id=41bde39f47f043acba3c006222877b68)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.182422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.247435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41bde39f47f043acba3c006222877b68', 'request_id': 'req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.256922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41bde39f47f043acba3c006222877b68', 'request_id': 'req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.269873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41bde39f47f043acba3c006222877b68', 'request_id': 'req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.290946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] The token cache is being invalidate because project 41bde39f47f043acba3c006222877b68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:20.291323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.292035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fa5a78e9-7ed9-4839-a2e4-c7332961c08f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.293659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1173/2338] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:20 2022] DELETE /identity/v3/projects/41bde39f47f043acba3c006222877b68 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:20.299301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745835e-eb6e-4dca-a31a-71abcce76abf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.300051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745835e-eb6e-4dca-a31a-71abcce76abf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.300419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745835e-eb6e-4dca-a31a-71abcce76abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.322347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b745835e-eb6e-4dca-a31a-71abcce76abf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:20.323580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b745835e-eb6e-4dca-a31a-71abcce76abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.394493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1166/2339] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:20.402836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab95332-34a1-46b1-b297-6738a9025b1b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.403325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab95332-34a1-46b1-b297-6738a9025b1b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.403599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab95332-34a1-46b1-b297-6738a9025b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.414303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745835e-eb6e-4dca-a31a-71abcce76abf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.415938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b745835e-eb6e-4dca-a31a-71abcce76abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.428745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9ab95332-34a1-46b1-b297-6738a9025b1b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:20.430135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745835e-eb6e-4dca-a31a-71abcce76abf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.430267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9ab95332-34a1-46b1-b297-6738a9025b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.434702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.435484 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.435856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.436221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] PATH_INFO: `/v3/projects/f93322b3b1c64fd3b2037d4ee2af8202` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.437612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93322b3b1c64fd3b2037d4ee2af8202)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.440021 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.444041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1174/2340] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:20.448048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab95332-34a1-46b1-b297-6738a9025b1b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.457435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab95332-34a1-46b1-b297-6738a9025b1b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.460083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:20.462430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.463069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.463413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.463799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.464435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:20.464707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.466791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.480337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1175/2341] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:43:20.485703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.494856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93322b3b1c64fd3b2037d4ee2af8202', 'request_id': 'req-fc208b34-f4e0-4683-824b-7b16472155bc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.496276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b065620-55df-4514-ac7c-7b2b63c6f893 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.500904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93322b3b1c64fd3b2037d4ee2af8202', 'request_id': 'req-fc208b34-f4e0-4683-824b-7b16472155bc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.507271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93322b3b1c64fd3b2037d4ee2af8202', 'request_id': 'req-fc208b34-f4e0-4683-824b-7b16472155bc'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.526999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] The token cache is being invalidate because project f93322b3b1c64fd3b2037d4ee2af8202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:20.527316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.527948 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.528535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:20.530505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1167/2342] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:20 2022] DELETE /identity/v3/projects/f93322b3b1c64fd3b2037d4ee2af8202 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:20.535105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.536531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.566754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.567468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.567681 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.567951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.568611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.570619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.599277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1176/2343] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:20 2022] POST /identity/v3/projects => generated 361 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:20.603613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.604792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208b34-f4e0-4683-824b-7b16472155bc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.605315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.666122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.666843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.667128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.667488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] PATH_INFO: `/v3/users/930376f7040447418b2573dbd27b9dbb` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.668660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=930376f7040447418b2573dbd27b9dbb)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.669836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407c89d9-3218-49d4-ab7a-e0370ff00c2f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.670644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.675204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299a41a-9777-4643-a8be-b2349c89df89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.675982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299a41a-9777-4643-a8be-b2349c89df89 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.677080 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299a41a-9777-4643-a8be-b2349c89df89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.677355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299a41a-9777-4643-a8be-b2349c89df89 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.678035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299a41a-9777-4643-a8be-b2349c89df89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.679978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299a41a-9777-4643-a8be-b2349c89df89 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.706942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1177/2344] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:20 2022] POST /identity/v3/users => generated 281 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:20.711112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299a41a-9777-4643-a8be-b2349c89df89 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.721511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '930376f7040447418b2573dbd27b9dbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.722203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299a41a-9777-4643-a8be-b2349c89df89 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.725929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:20.727792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.728555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.728852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.729119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.729227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '930376f7040447418b2573dbd27b9dbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.729794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.730007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '930376f7040447418b2573dbd27b9dbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.731552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.739202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '930376f7040447418b2573dbd27b9dbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.743178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1178/2345] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:20.748676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.750562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.756267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '930376f7040447418b2573dbd27b9dbb'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:20.762765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1168/2346] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:20 2022] DELETE /identity/v3/users/930376f7040447418b2573dbd27b9dbb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:20.769275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbd034-2fa6-4398-828a-985a0367149a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.831907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9355ce-45be-4f96-9228-61ba55f3b8c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.832655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9355ce-45be-4f96-9228-61ba55f3b8c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.832655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9355ce-45be-4f96-9228-61ba55f3b8c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.832980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9355ce-45be-4f96-9228-61ba55f3b8c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.833698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9355ce-45be-4f96-9228-61ba55f3b8c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.835346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9355ce-45be-4f96-9228-61ba55f3b8c3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.846617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1179/2347] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:20 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:20.850849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:20.861360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2927e1bd-1301-46bc-a5f5-08f4839d9ddd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:20.866656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:20.867312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.867671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.867981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] PATH_INFO: `/v3/projects/d581353dcebe4beb9cb6adecab544479/users/df090fe8b44546c1a73dcf73b4e8bc3f/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.884426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d581353dcebe4beb9cb6adecab544479, user_id=df090fe8b44546c1a73dcf73b4e8bc3f, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:20.887701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:20.907544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1169/2348] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:20 2022] PUT /identity/v3/projects/d581353dcebe4beb9cb6adecab544479/users/df090fe8b44546c1a73dcf73b4e8bc3f/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:20.920381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff2ecf-1808-46b4-92c8-2450d61126eb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:20.920761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff2ecf-1808-46b4-92c8-2450d61126eb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:20.921144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cff2ecf-1808-46b4-92c8-2450d61126eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:20.956930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0cff2ecf-1808-46b4-92c8-2450d61126eb None None] MFA Rules not processed for user `df090fe8b44546c1a73dcf73b4e8bc3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:20.958208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0cff2ecf-1808-46b4-92c8-2450d61126eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:20.971908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:21.077922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.080124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.116973 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1180/2349] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:20 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:21.192109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7191d-9905-41f8-9326-e857dd4473e4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.198771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.199756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.199936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.200203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] PATH_INFO: `/v3/projects/c0cfa24b5b7f47aab13b4bf795f65881` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.201508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0cfa24b5b7f47aab13b4bf795f65881)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.204168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.239348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff2ecf-1808-46b4-92c8-2450d61126eb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.240738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0cff2ecf-1808-46b4-92c8-2450d61126eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.271947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0cfa24b5b7f47aab13b4bf795f65881', 'request_id': 'req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.278220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0cfa24b5b7f47aab13b4bf795f65881', 'request_id': 'req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.285324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0cfa24b5b7f47aab13b4bf795f65881', 'request_id': 'req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.305572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] The token cache is being invalidate because project c0cfa24b5b7f47aab13b4bf795f65881 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:21.305907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.306101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cff2ecf-1808-46b4-92c8-2450d61126eb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.306575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-146f8fcf-6dd0-4d59-9693-3ce1fa9efc24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.308146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1170/2350] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:21 2022] DELETE /identity/v3/projects/c0cfa24b5b7f47aab13b4bf795f65881 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:21.313367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e40d8c-df0b-40bd-af08-e848c66853bf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.313745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e40d8c-df0b-40bd-af08-e848c66853bf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.314095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e40d8c-df0b-40bd-af08-e848c66853bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.334137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b9e40d8c-df0b-40bd-af08-e848c66853bf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:21.335426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b9e40d8c-df0b-40bd-af08-e848c66853bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.356798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.357337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.357611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.357905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.359808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.366182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:21.395229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1171/2351] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:21.403019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87304c-adc6-45b3-bcaa-8221c11de5f5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.403343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87304c-adc6-45b3-bcaa-8221c11de5f5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.403582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87304c-adc6-45b3-bcaa-8221c11de5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.420846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.422351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.425135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0f87304c-adc6-45b3-bcaa-8221c11de5f5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:21.426386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0f87304c-adc6-45b3-bcaa-8221c11de5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.431597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1181/2352] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:21.437675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1172/2353] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:21.439458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.449226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87304c-adc6-45b3-bcaa-8221c11de5f5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.451344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0f87304c-adc6-45b3-bcaa-8221c11de5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.458529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c9588b-04c8-4321-8a93-9c4cd3b3a134 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.463801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.464914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.465332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.466111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.466894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:21.468204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.470277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.478863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87304c-adc6-45b3-bcaa-8221c11de5f5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.488298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.489310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.490173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.490799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] PATH_INFO: `/v3/users/df090fe8b44546c1a73dcf73b4e8bc3f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.493525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df090fe8b44546c1a73dcf73b4e8bc3f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.493699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1182/2354] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:43:21.498480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.498981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.525173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a184990-6322-46c1-8747-f3ab3c70f454 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.530222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e8c20-834b-46cc-87a3-333c02a7eb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.530888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e8c20-834b-46cc-87a3-333c02a7eb80 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.531220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e8c20-834b-46cc-87a3-333c02a7eb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.531556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e8c20-834b-46cc-87a3-333c02a7eb80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.532260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e8c20-834b-46cc-87a3-333c02a7eb80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.534445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e8c20-834b-46cc-87a3-333c02a7eb80 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.550194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:21.568562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1183/2355] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:21 2022] POST /identity/v3/projects => generated 367 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:21.571268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df090fe8b44546c1a73dcf73b4e8bc3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.572936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e8c20-834b-46cc-87a3-333c02a7eb80 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.578607 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df090fe8b44546c1a73dcf73b4e8bc3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.579501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df090fe8b44546c1a73dcf73b4e8bc3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.592035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df090fe8b44546c1a73dcf73b4e8bc3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.612866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1da8ef44-1f6c-4c9a-8d7a-219003a04129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df090fe8b44546c1a73dcf73b4e8bc3f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:21.625752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1173/2356] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:21 2022] DELETE /identity/v3/users/df090fe8b44546c1a73dcf73b4e8bc3f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:21.627968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3795c7-bb58-4859-aafd-bb996d9ab79b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.628300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3795c7-bb58-4859-aafd-bb996d9ab79b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.628606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3795c7-bb58-4859-aafd-bb996d9ab79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.650891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e8c20-834b-46cc-87a3-333c02a7eb80 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.652702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3a3795c7-bb58-4859-aafd-bb996d9ab79b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:21.654070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3a3795c7-bb58-4859-aafd-bb996d9ab79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.708811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.709585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.709910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.710217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.710904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.713245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.716284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1174/2357] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:21.726209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3795c7-bb58-4859-aafd-bb996d9ab79b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.727738 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3a3795c7-bb58-4859-aafd-bb996d9ab79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.739961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1184/2358] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:21 2022] POST /identity/v3/users => generated 284 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:21.741081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.743382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.756890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b1ea4-f4c8-4690-a4ef-1c237306b7ca admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.761662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c44e7-7d73-48de-87ed-170bf7671bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.762255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c44e7-7d73-48de-87ed-170bf7671bff admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.762552 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c44e7-7d73-48de-87ed-170bf7671bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.762816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0c44e7-7d73-48de-87ed-170bf7671bff admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.763449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c44e7-7d73-48de-87ed-170bf7671bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.765100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0c44e7-7d73-48de-87ed-170bf7671bff admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.775127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1185/2359] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:21.775805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:21.779151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c44e7-7d73-48de-87ed-170bf7671bff admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.789084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0c44e7-7d73-48de-87ed-170bf7671bff admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.790286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3795c7-bb58-4859-aafd-bb996d9ab79b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.793900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae816e4-d711-46ec-9bac-676ecc8150c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.794504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae816e4-d711-46ec-9bac-676ecc8150c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.794773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae816e4-d711-46ec-9bac-676ecc8150c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.795037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161732b2-d203-4e49-994f-e7cf67e0698b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.795176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae816e4-d711-46ec-9bac-676ecc8150c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.795691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161732b2-d203-4e49-994f-e7cf67e0698b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.795802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae816e4-d711-46ec-9bac-676ecc8150c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.795964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161732b2-d203-4e49-994f-e7cf67e0698b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.796233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161732b2-d203-4e49-994f-e7cf67e0698b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.797289 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae816e4-d711-46ec-9bac-676ecc8150c6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.805896 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161732b2-d203-4e49-994f-e7cf67e0698b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.807374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161732b2-d203-4e49-994f-e7cf67e0698b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.808197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1186/2360] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:21.812236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae816e4-d711-46ec-9bac-676ecc8150c6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:21.818778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1175/2361] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:21.823790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae816e4-d711-46ec-9bac-676ecc8150c6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:21.828978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb334dea-2ba1-4dec-bd3e-2080b199b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:21.829746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb334dea-2ba1-4dec-bd3e-2080b199b71e admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.830013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb334dea-2ba1-4dec-bd3e-2080b199b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.830310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb334dea-2ba1-4dec-bd3e-2080b199b71e admin admin] PATH_INFO: `/v3/projects/749bdca4058c4657a6ee31ad9df556ea/users/95374714c9624ea0af42aeffdc734613/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.846539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb334dea-2ba1-4dec-bd3e-2080b199b71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=749bdca4058c4657a6ee31ad9df556ea, user_id=95374714c9624ea0af42aeffdc734613, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:21.849246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb334dea-2ba1-4dec-bd3e-2080b199b71e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:21.865793 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1187/2362] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:21 2022] PUT /identity/v3/projects/749bdca4058c4657a6ee31ad9df556ea/users/95374714c9624ea0af42aeffdc734613/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:21.879482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc103acd-8995-4292-9531-073155f9f32d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:21.879767 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc103acd-8995-4292-9531-073155f9f32d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:21.880036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc103acd-8995-4292-9531-073155f9f32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:21.913610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cc103acd-8995-4292-9531-073155f9f32d None None] MFA Rules not processed for user `95374714c9624ea0af42aeffdc734613`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:21.914958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cc103acd-8995-4292-9531-073155f9f32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:21.998043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1176/2363] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:22.002754 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb334dea-2ba1-4dec-bd3e-2080b199b71e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.058623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:22.084383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb334dea-2ba1-4dec-bd3e-2080b199b71e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.097198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.098107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.098528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.098939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.099849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.101950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.133345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1188/2364] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:22 2022] POST /identity/v3/projects => generated 369 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:22.138735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc103acd-8995-4292-9531-073155f9f32d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.142720 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.145101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:22.151277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc103acd-8995-4292-9531-073155f9f32d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.238423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d6872-4067-4c5f-a0ab-f675ea07ff54 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.240892 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2767b-9c41-42f1-a142-1f0a599656ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.241604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2767b-9c41-42f1-a142-1f0a599656ab admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.241912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2767b-9c41-42f1-a142-1f0a599656ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.242181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2767b-9c41-42f1-a142-1f0a599656ab admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.243006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2767b-9c41-42f1-a142-1f0a599656ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.245015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2767b-9c41-42f1-a142-1f0a599656ab admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.254631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.255853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.256306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.256576 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] PATH_INFO: `/v3/projects/d581353dcebe4beb9cb6adecab544479` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.258461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d581353dcebe4beb9cb6adecab544479)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.260866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.276729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1177/2365] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:22 2022] POST /identity/v3/users => generated 291 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:22.277967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2767b-9c41-42f1-a142-1f0a599656ab admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.288554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2767b-9c41-42f1-a142-1f0a599656ab admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.296090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df37155-f162-4229-893b-e8abccd482d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.296699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df37155-f162-4229-893b-e8abccd482d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.297028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df37155-f162-4229-893b-e8abccd482d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.297300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df37155-f162-4229-893b-e8abccd482d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.298015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df37155-f162-4229-893b-e8abccd482d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.299610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df37155-f162-4229-893b-e8abccd482d3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.310107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1178/2366] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:22.314288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df37155-f162-4229-893b-e8abccd482d3 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.325027 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df37155-f162-4229-893b-e8abccd482d3 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.335776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd581353dcebe4beb9cb6adecab544479', 'request_id': 'req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:22.342255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd581353dcebe4beb9cb6adecab544479', 'request_id': 'req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:22.350317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd581353dcebe4beb9cb6adecab544479', 'request_id': 'req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:22.364805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] The token cache is being invalidate because project d581353dcebe4beb9cb6adecab544479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:22.365088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:22.365743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7c9f5a8f-06ba-490b-96ef-c12b8a083fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:22.367415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1189/2367] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:22 2022] DELETE /identity/v3/projects/d581353dcebe4beb9cb6adecab544479 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:22.376336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a21a1b-b0c1-4a00-be01-b3fab436ff18 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.376550 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a21a1b-b0c1-4a00-be01-b3fab436ff18 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.376794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a21a1b-b0c1-4a00-be01-b3fab436ff18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.382832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:22.394167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.394939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.395269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.395586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] PATH_INFO: `/v3/projects/6fe4621951fb4f43b52056559ebcb41e/users/c939e81009414f228f71651dbabba322/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.397803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-35a21a1b-b0c1-4a00-be01-b3fab436ff18 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:22.399068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-35a21a1b-b0c1-4a00-be01-b3fab436ff18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:22.408071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1190/2368] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:22.411733 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe4621951fb4f43b52056559ebcb41e, user_id=c939e81009414f228f71651dbabba322, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.414473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.415958 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9903a-3d0e-48a6-8d20-856c96b118de None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.416379 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9903a-3d0e-48a6-8d20-856c96b118de None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.416685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9903a-3d0e-48a6-8d20-856c96b118de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.430231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1179/2369] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:22 2022] PUT /identity/v3/projects/6fe4621951fb4f43b52056559ebcb41e/users/c939e81009414f228f71651dbabba322/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:22.434103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.435725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:22.437986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3bd9903a-3d0e-48a6-8d20-856c96b118de None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:22.439212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3bd9903a-3d0e-48a6-8d20-856c96b118de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:22.490122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:22.506804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1191/2370] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:22.511529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9903a-3d0e-48a6-8d20-856c96b118de None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.518151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb897fa7-08d7-4f58-b61c-06569017e329 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.523817 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9903a-3d0e-48a6-8d20-856c96b118de None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.524153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3f6d-53bb-4613-9afd-15e827c6ab51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.524849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3f6d-53bb-4613-9afd-15e827c6ab51 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.525227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3f6d-53bb-4613-9afd-15e827c6ab51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.525558 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e3f6d-53bb-4613-9afd-15e827c6ab51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.526358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e3f6d-53bb-4613-9afd-15e827c6ab51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.532343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.532985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.533216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.533493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.534198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:22.534540 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.536169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.539616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e3f6d-53bb-4613-9afd-15e827c6ab51 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.554297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1192/2372] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:43:22.554726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1180/2372] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:22.556802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.562817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3f6d-53bb-4613-9afd-15e827c6ab51 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.569451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76d7d50-0f88-4940-a9b4-b5a462ae0b35 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.574414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954befac-b6c9-4886-8001-5fed299115f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.575062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954befac-b6c9-4886-8001-5fed299115f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.575339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954befac-b6c9-4886-8001-5fed299115f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.575610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954befac-b6c9-4886-8001-5fed299115f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.576261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954befac-b6c9-4886-8001-5fed299115f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.578691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954befac-b6c9-4886-8001-5fed299115f4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.587265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e3f6d-53bb-4613-9afd-15e827c6ab51 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.593166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a463878-b73d-486c-ad56-60c29cf62029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.593586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a463878-b73d-486c-ad56-60c29cf62029 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.593913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a463878-b73d-486c-ad56-60c29cf62029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.594148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a463878-b73d-486c-ad56-60c29cf62029 admin admin] PATH_INFO: `/v3/projects/6fe4621951fb4f43b52056559ebcb41e/users/c939e81009414f228f71651dbabba322/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.596102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a463878-b73d-486c-ad56-60c29cf62029 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fe4621951fb4f43b52056559ebcb41e, user_id=c939e81009414f228f71651dbabba322, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.599489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a463878-b73d-486c-ad56-60c29cf62029 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.613332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1181/2373] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:22 2022] PUT /identity/v3/projects/6fe4621951fb4f43b52056559ebcb41e/users/c939e81009414f228f71651dbabba322/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:22.618295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1193/2374] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:22 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:22.622495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a463878-b73d-486c-ad56-60c29cf62029 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.628306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c61805-9532-46d1-a559-f0cc4aab10ad None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.628513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c61805-9532-46d1-a559-f0cc4aab10ad None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.628778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c61805-9532-46d1-a559-f0cc4aab10ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.663555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-31c61805-9532-46d1-a559-f0cc4aab10ad None None] MFA Rules not processed for user `c939e81009414f228f71651dbabba322`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:22.665060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-31c61805-9532-46d1-a559-f0cc4aab10ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:22.699706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a463878-b73d-486c-ad56-60c29cf62029 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.752558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1194/2375] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:22.765849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c61805-9532-46d1-a559-f0cc4aab10ad None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.767986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-31c61805-9532-46d1-a559-f0cc4aab10ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:22.787376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d39bf-6ecb-41ad-a2f8-13e7d962f85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.788259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d39bf-6ecb-41ad-a2f8-13e7d962f85a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.788370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d39bf-6ecb-41ad-a2f8-13e7d962f85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.788628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d39bf-6ecb-41ad-a2f8-13e7d962f85a admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.789886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d39bf-6ecb-41ad-a2f8-13e7d962f85a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.792670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d39bf-6ecb-41ad-a2f8-13e7d962f85a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.814866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:22.834862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1182/2376] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:22 2022] POST /identity/v3/users => generated 290 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:22.839020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d39bf-6ecb-41ad-a2f8-13e7d962f85a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.850374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d39bf-6ecb-41ad-a2f8-13e7d962f85a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.855124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0195e1-d98a-4168-b153-b0c84568a560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.855725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0195e1-d98a-4168-b153-b0c84568a560 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.855998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0195e1-d98a-4168-b153-b0c84568a560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.856311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0195e1-d98a-4168-b153-b0c84568a560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.856958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0195e1-d98a-4168-b153-b0c84568a560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.858727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0195e1-d98a-4168-b153-b0c84568a560 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.864859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c61805-9532-46d1-a559-f0cc4aab10ad None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.868546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587c1ae-2b61-4cd1-8fc3-10d3dbd12b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.869171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c1ae-2b61-4cd1-8fc3-10d3dbd12b57 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.869291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1183/2377] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:22.869451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c1ae-2b61-4cd1-8fc3-10d3dbd12b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.869763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a587c1ae-2b61-4cd1-8fc3-10d3dbd12b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.872863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0195e1-d98a-4168-b153-b0c84568a560 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.879158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587c1ae-2b61-4cd1-8fc3-10d3dbd12b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.880541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a587c1ae-2b61-4cd1-8fc3-10d3dbd12b57 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.884090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0195e1-d98a-4168-b153-b0c84568a560 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.888628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.889201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.889920 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.890195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.890842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.892920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1195/2378] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:22.895153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.907880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1184/2379] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:22.912124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587c1ae-2b61-4cd1-8fc3-10d3dbd12b57 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.927787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a587c1ae-2b61-4cd1-8fc3-10d3dbd12b57 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.934190 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398cae2f-029e-4b94-865c-09fe17d589d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:22.934975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cae2f-029e-4b94-865c-09fe17d589d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.935359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cae2f-029e-4b94-865c-09fe17d589d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.935600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398cae2f-029e-4b94-865c-09fe17d589d0 admin admin] PATH_INFO: `/v3/projects/49adc8da67bf4529a349a2bd79e87e07/users/bc27904d964e43438c32876b8a1b202d/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:22.952619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:22.954402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:22.955635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cae2f-029e-4b94-865c-09fe17d589d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49adc8da67bf4529a349a2bd79e87e07, user_id=bc27904d964e43438c32876b8a1b202d, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:22.958399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398cae2f-029e-4b94-865c-09fe17d589d0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:22.967169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd217c0-deae-4f7b-8b4f-d2e4c4c3287a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:22.974263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1196/2380] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:22 2022] PUT /identity/v3/projects/49adc8da67bf4529a349a2bd79e87e07/users/bc27904d964e43438c32876b8a1b202d/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:22.979347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42153-0426-4edb-b41e-51fdb90acdb0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:22.979736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42153-0426-4edb-b41e-51fdb90acdb0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:22.980146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c42153-0426-4edb-b41e-51fdb90acdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:23.001483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-43c42153-0426-4edb-b41e-51fdb90acdb0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:23.003282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-43c42153-0426-4edb-b41e-51fdb90acdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:23.016915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:23.023897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f6223-73ce-4cd9-b462-a21c3a675a58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:23.024805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f6223-73ce-4cd9-b462-a21c3a675a58 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:23.025621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f6223-73ce-4cd9-b462-a21c3a675a58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:23.026215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f6223-73ce-4cd9-b462-a21c3a675a58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:23.029146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c39f6223-73ce-4cd9-b462-a21c3a675a58 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:23.069281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1197/2381] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:23.080427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c42153-0426-4edb-b41e-51fdb90acdb0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:23.081901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-43c42153-0426-4edb-b41e-51fdb90acdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:23.093916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f6223-73ce-4cd9-b462-a21c3a675a58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:23.095370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39f6223-73ce-4cd9-b462-a21c3a675a58 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:23.105442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1185/2382] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:23.154985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c42153-0426-4edb-b41e-51fdb90acdb0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:23.162010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a9799-8c1a-4235-ae4c-a258374bec88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:23.162849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a9799-8c1a-4235-ae4c-a258374bec88 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:23.163079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a9799-8c1a-4235-ae4c-a258374bec88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:23.163411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a9799-8c1a-4235-ae4c-a258374bec88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:23.173310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a9799-8c1a-4235-ae4c-a258374bec88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:23.174807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a9799-8c1a-4235-ae4c-a258374bec88 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:23.184416 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1198/2383] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:23.957325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e82bb3-224d-45c9-91f5-0fcf57666450 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:23.957874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e82bb3-224d-45c9-91f5-0fcf57666450 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:23.957969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e82bb3-224d-45c9-91f5-0fcf57666450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:23.986914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-84e82bb3-224d-45c9-91f5-0fcf57666450 None None] MFA Rules not processed for user `bc27904d964e43438c32876b8a1b202d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:23.988399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-84e82bb3-224d-45c9-91f5-0fcf57666450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:24.057145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1186/2384] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:24.068260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a9799-8c1a-4235-ae4c-a258374bec88 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:24.077835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a9799-8c1a-4235-ae4c-a258374bec88 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:24.081494 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc86a85-369a-481f-8263-55eb82099553 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:24.082133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc86a85-369a-481f-8263-55eb82099553 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:24.082406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc86a85-369a-481f-8263-55eb82099553 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:24.082676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc86a85-369a-481f-8263-55eb82099553 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:24.091551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc86a85-369a-481f-8263-55eb82099553 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:24.093054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc86a85-369a-481f-8263-55eb82099553 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:24.102118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1199/2385] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:26.680442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e82bb3-224d-45c9-91f5-0fcf57666450 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:26.691916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e82bb3-224d-45c9-91f5-0fcf57666450 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:26.696638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:26.697302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:26.697577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:26.697907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] PATH_INFO: `/v3/users/95374714c9624ea0af42aeffdc734613` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:26.699149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95374714c9624ea0af42aeffdc734613)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:26.701084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:26.714220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:26.752419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95374714c9624ea0af42aeffdc734613'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:26.759157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95374714c9624ea0af42aeffdc734613'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:26.759803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95374714c9624ea0af42aeffdc734613'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:26.768186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95374714c9624ea0af42aeffdc734613'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:26.779954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95374714c9624ea0af42aeffdc734613'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:26.786582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1187/2386] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:26 2022] DELETE /identity/v3/users/95374714c9624ea0af42aeffdc734613 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:26.791095 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a476d-23a3-4ce2-88a4-321cb575d333 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:26.791609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a476d-23a3-4ce2-88a4-321cb575d333 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:26.791609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59a476d-23a3-4ce2-88a4-321cb575d333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:26.811378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f59a476d-23a3-4ce2-88a4-321cb575d333 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:26.812609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f59a476d-23a3-4ce2-88a4-321cb575d333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:26.891181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1200/2387] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:26.903361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:26.905690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:26.971143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f7cf5-0013-487b-a988-a455b4e48613 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:26.975004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce882f-d04e-4e4c-8897-89ff16e0d9f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:26.975650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce882f-d04e-4e4c-8897-89ff16e0d9f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:26.975939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce882f-d04e-4e4c-8897-89ff16e0d9f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:26.976186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce882f-d04e-4e4c-8897-89ff16e0d9f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:26.986015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce882f-d04e-4e4c-8897-89ff16e0d9f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:26.987463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce882f-d04e-4e4c-8897-89ff16e0d9f1 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:26.997120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1188/2388] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:27.275636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a476d-23a3-4ce2-88a4-321cb575d333 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:27.277340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f59a476d-23a3-4ce2-88a4-321cb575d333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:27.291039 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59a476d-23a3-4ce2-88a4-321cb575d333 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:27.296878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:27.297530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:27.297828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:27.298064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] PATH_INFO: `/v3/projects/749bdca4058c4657a6ee31ad9df556ea` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:27.299304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=749bdca4058c4657a6ee31ad9df556ea)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:27.301329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:27.351466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:27.354795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '749bdca4058c4657a6ee31ad9df556ea', 'request_id': 'req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.361992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '749bdca4058c4657a6ee31ad9df556ea', 'request_id': 'req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.368750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '749bdca4058c4657a6ee31ad9df556ea', 'request_id': 'req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.382741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] The token cache is being invalidate because project 749bdca4058c4657a6ee31ad9df556ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:27.383029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.383671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.385354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1201/2389] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:27 2022] DELETE /identity/v3/projects/749bdca4058c4657a6ee31ad9df556ea => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:27.389457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce882f-d04e-4e4c-8897-89ff16e0d9f1 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:27.399440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce882f-d04e-4e4c-8897-89ff16e0d9f1 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:27.456949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:27.457612 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:27.457893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:27.458162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] PATH_INFO: `/v3/users/c939e81009414f228f71651dbabba322` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:27.459285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c939e81009414f228f71651dbabba322)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:27.461321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:27.499822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:27.512477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c939e81009414f228f71651dbabba322'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.519747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c939e81009414f228f71651dbabba322'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.520425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c939e81009414f228f71651dbabba322'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.530798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c939e81009414f228f71651dbabba322'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.544407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f680dc8a-5017-415c-80f7-639e84b76c2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c939e81009414f228f71651dbabba322'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.550864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1189/2390] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:27 2022] DELETE /identity/v3/users/c939e81009414f228f71651dbabba322 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:27.829162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:27.830703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:27.847525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cdf80f-f9b0-42de-b6c7-466ab3ec9fa3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:27.914704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:27.915349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:27.915630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:27.915897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] PATH_INFO: `/v3/projects/6fe4621951fb4f43b52056559ebcb41e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:27.917021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fe4621951fb4f43b52056559ebcb41e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:27.920287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:27.969607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fe4621951fb4f43b52056559ebcb41e', 'request_id': 'req-7bfacc76-c483-44d7-be04-dfb4b0950ffb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.975008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fe4621951fb4f43b52056559ebcb41e', 'request_id': 'req-7bfacc76-c483-44d7-be04-dfb4b0950ffb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.981073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fe4621951fb4f43b52056559ebcb41e', 'request_id': 'req-7bfacc76-c483-44d7-be04-dfb4b0950ffb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.995032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] The token cache is being invalidate because project 6fe4621951fb4f43b52056559ebcb41e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:27.995316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.996020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7bfacc76-c483-44d7-be04-dfb4b0950ffb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:27.997742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1202/2391] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:27 2022] DELETE /identity/v3/projects/6fe4621951fb4f43b52056559ebcb41e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:28.002805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff3342-5591-4a96-9954-eeccd0aa1ab7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.003303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff3342-5591-4a96-9954-eeccd0aa1ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.003403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ff3342-5591-4a96-9954-eeccd0aa1ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.023420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f2ff3342-5591-4a96-9954-eeccd0aa1ab7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:28.024758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f2ff3342-5591-4a96-9954-eeccd0aa1ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:28.087625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1190/2392] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:28.096451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845c576-9505-4fa1-9439-83433f5781b3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.096733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845c576-9505-4fa1-9439-83433f5781b3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.097007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f845c576-9505-4fa1-9439-83433f5781b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.119670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f845c576-9505-4fa1-9439-83433f5781b3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:28.120814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f845c576-9505-4fa1-9439-83433f5781b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:28.131740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1203/2393] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:28.137310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff3342-5591-4a96-9954-eeccd0aa1ab7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.147491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ff3342-5591-4a96-9954-eeccd0aa1ab7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.152225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.152827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.153096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.153366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.154051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:28.154351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.155950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.167325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1191/2394] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:43:28.172051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845c576-9505-4fa1-9439-83433f5781b3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.181663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f845c576-9505-4fa1-9439-83433f5781b3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.187130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a1e3f-cf08-402f-94cb-5eb0fc74256f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.187737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a1e3f-cf08-402f-94cb-5eb0fc74256f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.188007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a1e3f-cf08-402f-94cb-5eb0fc74256f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.188247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92a1e3f-cf08-402f-94cb-5eb0fc74256f admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.188884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a1e3f-cf08-402f-94cb-5eb0fc74256f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.190849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92a1e3f-cf08-402f-94cb-5eb0fc74256f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.221259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1204/2395] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:28 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:28.226330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.237123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df50be00-20d3-4e9e-bd14-c3011871b0d6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.293750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f452-cd5b-4ba4-b71f-7cb7bf2ae74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.295392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f452-cd5b-4ba4-b71f-7cb7bf2ae74f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.295849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f452-cd5b-4ba4-b71f-7cb7bf2ae74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.296120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57f452-cd5b-4ba4-b71f-7cb7bf2ae74f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.297294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57f452-cd5b-4ba4-b71f-7cb7bf2ae74f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.299480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b57f452-cd5b-4ba4-b71f-7cb7bf2ae74f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.322837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1192/2396] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:28 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:28.327284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a1e3f-cf08-402f-94cb-5eb0fc74256f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.337997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92a1e3f-cf08-402f-94cb-5eb0fc74256f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.344036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53565e-12a8-4d1b-b106-0164c22b2862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.344759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53565e-12a8-4d1b-b106-0164c22b2862 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.345339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53565e-12a8-4d1b-b106-0164c22b2862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.345609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba53565e-12a8-4d1b-b106-0164c22b2862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.347129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53565e-12a8-4d1b-b106-0164c22b2862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.349536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba53565e-12a8-4d1b-b106-0164c22b2862 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.360127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1205/2397] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:28.366067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f452-cd5b-4ba4-b71f-7cb7bf2ae74f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.377941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57f452-cd5b-4ba4-b71f-7cb7bf2ae74f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.383427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436a7533-6883-42ef-bc50-69582ae759b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.384075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a7533-6883-42ef-bc50-69582ae759b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.384373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a7533-6883-42ef-bc50-69582ae759b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.384642 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436a7533-6883-42ef-bc50-69582ae759b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.385324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a7533-6883-42ef-bc50-69582ae759b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.386998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436a7533-6883-42ef-bc50-69582ae759b3 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.398410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1193/2398] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:28.402766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53565e-12a8-4d1b-b106-0164c22b2862 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.413306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba53565e-12a8-4d1b-b106-0164c22b2862 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.418257 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109ec0c-aab5-4253-bb2e-9d72b447f8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.418964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109ec0c-aab5-4253-bb2e-9d72b447f8e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.419250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109ec0c-aab5-4253-bb2e-9d72b447f8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.419503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109ec0c-aab5-4253-bb2e-9d72b447f8e9 admin admin] PATH_INFO: `/v3/projects/4fe3e20edccd405ab0849c15e4766542/users/35aec7b9bdae4dd496e8337a5a6a30f2/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.435340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109ec0c-aab5-4253-bb2e-9d72b447f8e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fe3e20edccd405ab0849c15e4766542, user_id=35aec7b9bdae4dd496e8337a5a6a30f2, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.438217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109ec0c-aab5-4253-bb2e-9d72b447f8e9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.449086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:28.454224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1206/2399] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:28 2022] PUT /identity/v3/projects/4fe3e20edccd405ab0849c15e4766542/users/35aec7b9bdae4dd496e8337a5a6a30f2/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:28.467039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410dae3-1cf8-422f-aed0-a4fabb00dd83 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.467489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410dae3-1cf8-422f-aed0-a4fabb00dd83 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.467756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410dae3-1cf8-422f-aed0-a4fabb00dd83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.511941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6410dae3-1cf8-422f-aed0-a4fabb00dd83 None None] MFA Rules not processed for user `35aec7b9bdae4dd496e8337a5a6a30f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:28.513145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6410dae3-1cf8-422f-aed0-a4fabb00dd83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:28.518515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:28.584548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1194/2400] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:28.589154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109ec0c-aab5-4253-bb2e-9d72b447f8e9 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.667703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109ec0c-aab5-4253-bb2e-9d72b447f8e9 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.672570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915ca64-5ec4-4878-adf6-116f4041bf61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.673254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915ca64-5ec4-4878-adf6-116f4041bf61 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.673586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915ca64-5ec4-4878-adf6-116f4041bf61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.674181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b915ca64-5ec4-4878-adf6-116f4041bf61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.675556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915ca64-5ec4-4878-adf6-116f4041bf61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.678821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b915ca64-5ec4-4878-adf6-116f4041bf61 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.708105 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1207/2401] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:28 2022] POST /identity/v3/projects => generated 387 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:28.712408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410dae3-1cf8-422f-aed0-a4fabb00dd83 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.723890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410dae3-1cf8-422f-aed0-a4fabb00dd83 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.788071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acec23a-cbc8-4db5-88d6-dca37bca2922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.788676 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec23a-cbc8-4db5-88d6-dca37bca2922 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.788951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec23a-cbc8-4db5-88d6-dca37bca2922 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.789216 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acec23a-cbc8-4db5-88d6-dca37bca2922 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.789888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acec23a-cbc8-4db5-88d6-dca37bca2922 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.791843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acec23a-cbc8-4db5-88d6-dca37bca2922 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.819329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1195/2402] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:28 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:28.821270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915ca64-5ec4-4878-adf6-116f4041bf61 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.831846 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b915ca64-5ec4-4878-adf6-116f4041bf61 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.836403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1056c4-23eb-40dc-b16e-5d598e073bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.837035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1056c4-23eb-40dc-b16e-5d598e073bbd admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.837315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1056c4-23eb-40dc-b16e-5d598e073bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.837580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1056c4-23eb-40dc-b16e-5d598e073bbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.838250 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1056c4-23eb-40dc-b16e-5d598e073bbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.839877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1056c4-23eb-40dc-b16e-5d598e073bbd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.850633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1208/2403] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:28.854661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acec23a-cbc8-4db5-88d6-dca37bca2922 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.865151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acec23a-cbc8-4db5-88d6-dca37bca2922 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.870249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452a30b-4bd2-49af-82fe-0d5303c801e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.870937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452a30b-4bd2-49af-82fe-0d5303c801e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.871213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452a30b-4bd2-49af-82fe-0d5303c801e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.871490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2452a30b-4bd2-49af-82fe-0d5303c801e1 admin admin] PATH_INFO: `/v3/projects/dc29c72409b44af289205d42129d1e7c/users/f9ee365f564240308cf181ec599f7563/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.888307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452a30b-4bd2-49af-82fe-0d5303c801e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc29c72409b44af289205d42129d1e7c, user_id=f9ee365f564240308cf181ec599f7563, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.891113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2452a30b-4bd2-49af-82fe-0d5303c801e1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:28.906764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1196/2404] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:28 2022] PUT /identity/v3/projects/dc29c72409b44af289205d42129d1e7c/users/f9ee365f564240308cf181ec599f7563/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:28.910804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1056c4-23eb-40dc-b16e-5d598e073bbd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:28.921500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1056c4-23eb-40dc-b16e-5d598e073bbd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:28.992965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce27bba-fbc5-4756-8b6c-72d0581a6fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:28.993831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27bba-fbc5-4756-8b6c-72d0581a6fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:28.994360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27bba-fbc5-4756-8b6c-72d0581a6fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:28.994642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce27bba-fbc5-4756-8b6c-72d0581a6fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:28.995277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce27bba-fbc5-4756-8b6c-72d0581a6fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:28.996972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce27bba-fbc5-4756-8b6c-72d0581a6fb9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:29.007246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1209/2405] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:28 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:29.011383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452a30b-4bd2-49af-82fe-0d5303c801e1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:29.021512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2452a30b-4bd2-49af-82fe-0d5303c801e1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:29.026110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:29.026765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:29.027043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:29.027314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] PATH_INFO: `/v3/projects/dc29c72409b44af289205d42129d1e7c/users/f9ee365f564240308cf181ec599f7563/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:29.029199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc29c72409b44af289205d42129d1e7c, user_id=f9ee365f564240308cf181ec599f7563, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:29.031884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:29.041987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1197/2406] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:29 2022] PUT /identity/v3/projects/dc29c72409b44af289205d42129d1e7c/users/f9ee365f564240308cf181ec599f7563/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:29.052378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683c3eb-81c8-44d2-a389-fafcd973fa56 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:29.052663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683c3eb-81c8-44d2-a389-fafcd973fa56 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:29.052941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683c3eb-81c8-44d2-a389-fafcd973fa56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:29.081892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-d683c3eb-81c8-44d2-a389-fafcd973fa56 None None] MFA Rules not processed for user `f9ee365f564240308cf181ec599f7563`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:29.083125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d683c3eb-81c8-44d2-a389-fafcd973fa56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:29.144657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:29.154263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1210/2407] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:29.166108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:29.168044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:29.233663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082546d0-e9ad-4fb0-a23d-7bbb17eb524b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:29.238188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61138099-4999-43d1-992e-f610bb7e88b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:29.238800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61138099-4999-43d1-992e-f610bb7e88b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:29.239090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61138099-4999-43d1-992e-f610bb7e88b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:29.239334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61138099-4999-43d1-992e-f610bb7e88b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:29.251543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61138099-4999-43d1-992e-f610bb7e88b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:29.253001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61138099-4999-43d1-992e-f610bb7e88b6 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:29.263372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1198/2408] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:29 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:29.358012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683c3eb-81c8-44d2-a389-fafcd973fa56 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:29.423831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d683c3eb-81c8-44d2-a389-fafcd973fa56 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:29.428730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:29.429375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:29.429672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:29.429952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] PATH_INFO: `/v3/users/35aec7b9bdae4dd496e8337a5a6a30f2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:29.431151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35aec7b9bdae4dd496e8337a5a6a30f2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:29.433380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:29.489272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35aec7b9bdae4dd496e8337a5a6a30f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:29.496716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35aec7b9bdae4dd496e8337a5a6a30f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:29.497516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35aec7b9bdae4dd496e8337a5a6a30f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:29.507294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35aec7b9bdae4dd496e8337a5a6a30f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:29.526173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35aec7b9bdae4dd496e8337a5a6a30f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:29.532769 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1211/2409] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:29 2022] DELETE /identity/v3/users/35aec7b9bdae4dd496e8337a5a6a30f2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:29.543316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a5316-5f7e-4908-bca4-dc200be9ddd2 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:29.543537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a5316-5f7e-4908-bca4-dc200be9ddd2 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:29.543898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a5316-5f7e-4908-bca4-dc200be9ddd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:29.552385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:29.567481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4f0a5316-5f7e-4908-bca4-dc200be9ddd2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:29.568770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4f0a5316-5f7e-4908-bca4-dc200be9ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:29.632595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1199/2410] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:29.642624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:29.644069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:29.706547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1526e8-9b24-4431-8e4c-36696f785e14 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:29.710582 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82755a-4e29-4b79-a856-0a923d8957d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:29.711201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82755a-4e29-4b79-a856-0a923d8957d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:29.711532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82755a-4e29-4b79-a856-0a923d8957d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:29.711744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82755a-4e29-4b79-a856-0a923d8957d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:29.722229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82755a-4e29-4b79-a856-0a923d8957d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:29.723678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c82755a-4e29-4b79-a856-0a923d8957d0 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:29.733523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1212/2411] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:30.027664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a5316-5f7e-4908-bca4-dc200be9ddd2 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:30.027664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4f0a5316-5f7e-4908-bca4-dc200be9ddd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:30.046844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a5316-5f7e-4908-bca4-dc200be9ddd2 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:30.052456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:30.053115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:30.053443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:30.053662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] PATH_INFO: `/v3/projects/4fe3e20edccd405ab0849c15e4766542` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:30.054817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fe3e20edccd405ab0849c15e4766542)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:30.056719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:30.120391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fe3e20edccd405ab0849c15e4766542', 'request_id': 'req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.126518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fe3e20edccd405ab0849c15e4766542', 'request_id': 'req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.133477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fe3e20edccd405ab0849c15e4766542', 'request_id': 'req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.155720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] The token cache is being invalidate because project 4fe3e20edccd405ab0849c15e4766542 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:30.156011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.156647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.158355 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1200/2412] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:30 2022] DELETE /identity/v3/projects/4fe3e20edccd405ab0849c15e4766542 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:30.162700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82755a-4e29-4b79-a856-0a923d8957d0 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:30.174822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82755a-4e29-4b79-a856-0a923d8957d0 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:30.187928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:30.237944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:30.238739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:30.239019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:30.239286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] PATH_INFO: `/v3/users/f9ee365f564240308cf181ec599f7563` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:30.240406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9ee365f564240308cf181ec599f7563)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:30.242393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:30.307405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9ee365f564240308cf181ec599f7563'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.315663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9ee365f564240308cf181ec599f7563'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.316505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9ee365f564240308cf181ec599f7563'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.328753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9ee365f564240308cf181ec599f7563'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.343828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5509320-f3c0-4142-8f82-5169406e3559 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9ee365f564240308cf181ec599f7563'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.350981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1213/2413] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:30 2022] DELETE /identity/v3/users/f9ee365f564240308cf181ec599f7563 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:30.678901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:30.681150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:30.702699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c584b3-2aa6-4e47-aab5-43b0fcf445ea admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:30.796687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457b391-1d8b-4bff-bdc9-590b51f8035f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:30.797090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457b391-1d8b-4bff-bdc9-590b51f8035f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:30.797387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7457b391-1d8b-4bff-bdc9-590b51f8035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:30.812384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:30.813181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:30.813952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:30.814245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] PATH_INFO: `/v3/projects/dc29c72409b44af289205d42129d1e7c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:30.816319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc29c72409b44af289205d42129d1e7c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:30.818656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:30.820219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7457b391-1d8b-4bff-bdc9-590b51f8035f None None] MFA Rules not processed for user `bc27904d964e43438c32876b8a1b202d`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:30.821586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7457b391-1d8b-4bff-bdc9-590b51f8035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:30.882143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:30.911355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1214/2414] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:30.911760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc29c72409b44af289205d42129d1e7c', 'request_id': 'req-fac9aedc-a69e-4d3b-be86-44b022bc9bee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.920660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc29c72409b44af289205d42129d1e7c', 'request_id': 'req-fac9aedc-a69e-4d3b-be86-44b022bc9bee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.929222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc29c72409b44af289205d42129d1e7c', 'request_id': 'req-fac9aedc-a69e-4d3b-be86-44b022bc9bee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.968005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] The token cache is being invalidate because project dc29c72409b44af289205d42129d1e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:30.968272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.969421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fac9aedc-a69e-4d3b-be86-44b022bc9bee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:30.971351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1201/2415] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:30 2022] DELETE /identity/v3/projects/dc29c72409b44af289205d42129d1e7c => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:30.977885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61c4c1-1afa-45e0-b0be-c925125539ab None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:30.978486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61c4c1-1afa-45e0-b0be-c925125539ab None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:30.981946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c61c4c1-1afa-45e0-b0be-c925125539ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.011311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9c61c4c1-1afa-45e0-b0be-c925125539ab None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:31.013492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9c61c4c1-1afa-45e0-b0be-c925125539ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:31.063030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:31.097418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1215/2416] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:31.106171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee96b-4083-48ab-9f01-4125f0e58a79 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.106977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee96b-4083-48ab-9f01-4125f0e58a79 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.108219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ee96b-4083-48ab-9f01-4125f0e58a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.131843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-da7ee96b-4083-48ab-9f01-4125f0e58a79 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:31.132890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-da7ee96b-4083-48ab-9f01-4125f0e58a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:31.142071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1202/2417] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:31.146736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61c4c1-1afa-45e0-b0be-c925125539ab None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.156811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c61c4c1-1afa-45e0-b0be-c925125539ab None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.163180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.163829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.164172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.164471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.165131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:31.165482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.167741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.186523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1216/2418] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:43:31.191190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ee96b-4083-48ab-9f01-4125f0e58a79 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.201085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ee96b-4083-48ab-9f01-4125f0e58a79 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.207500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d6a87-881d-4bb2-a8f2-e107214b93bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.208183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d6a87-881d-4bb2-a8f2-e107214b93bb admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.208472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d6a87-881d-4bb2-a8f2-e107214b93bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.208764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d6a87-881d-4bb2-a8f2-e107214b93bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.209670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d6a87-881d-4bb2-a8f2-e107214b93bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.211619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d6a87-881d-4bb2-a8f2-e107214b93bb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.252626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1203/2419] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:31 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:31.257308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.269079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ccccaf-371a-48c8-8233-b16dd79a07da admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.340789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afae82-36ec-4f82-8a18-c95794ee9f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.341708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afae82-36ec-4f82-8a18-c95794ee9f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.342347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afae82-36ec-4f82-8a18-c95794ee9f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.342626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afae82-36ec-4f82-8a18-c95794ee9f77 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.343288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afae82-36ec-4f82-8a18-c95794ee9f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.345252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afae82-36ec-4f82-8a18-c95794ee9f77 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.375472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1217/2420] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:31 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:31.380224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d6a87-881d-4bb2-a8f2-e107214b93bb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.393024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d6a87-881d-4bb2-a8f2-e107214b93bb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.399073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259c8ee-bfa2-4f17-ace3-f59be22e905b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.399817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259c8ee-bfa2-4f17-ace3-f59be22e905b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.400105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259c8ee-bfa2-4f17-ace3-f59be22e905b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.400399 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259c8ee-bfa2-4f17-ace3-f59be22e905b admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.401038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259c8ee-bfa2-4f17-ace3-f59be22e905b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.417569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a259c8ee-bfa2-4f17-ace3-f59be22e905b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.433716 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1204/2421] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:31.439648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afae82-36ec-4f82-8a18-c95794ee9f77 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.451695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afae82-36ec-4f82-8a18-c95794ee9f77 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.460029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6158c36-15e2-4db6-8229-9ec8c597ac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.460766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158c36-15e2-4db6-8229-9ec8c597ac5a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.461279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158c36-15e2-4db6-8229-9ec8c597ac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.461738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6158c36-15e2-4db6-8229-9ec8c597ac5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.462405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6158c36-15e2-4db6-8229-9ec8c597ac5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.466429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6158c36-15e2-4db6-8229-9ec8c597ac5a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.483015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1218/2422] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:31.487195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259c8ee-bfa2-4f17-ace3-f59be22e905b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.501199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259c8ee-bfa2-4f17-ace3-f59be22e905b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.514233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718adfe-176d-4e61-9baf-feededf484fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.514930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718adfe-176d-4e61-9baf-feededf484fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.515208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718adfe-176d-4e61-9baf-feededf484fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.515524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c718adfe-176d-4e61-9baf-feededf484fe admin admin] PATH_INFO: `/v3/projects/b411fbf1d27b4e7f894845ab950825a7/users/b1dc16d53d4a407d875c93f3565e414b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.533805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718adfe-176d-4e61-9baf-feededf484fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=b411fbf1d27b4e7f894845ab950825a7, user_id=b1dc16d53d4a407d875c93f3565e414b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.536731 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c718adfe-176d-4e61-9baf-feededf484fe admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.552196 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1205/2423] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:31 2022] PUT /identity/v3/projects/b411fbf1d27b4e7f894845ab950825a7/users/b1dc16d53d4a407d875c93f3565e414b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:31.564642 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23bd62-8c58-4318-aac0-82c346d8a3e2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.564999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23bd62-8c58-4318-aac0-82c346d8a3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.565274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23bd62-8c58-4318-aac0-82c346d8a3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.594373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0e23bd62-8c58-4318-aac0-82c346d8a3e2 None None] MFA Rules not processed for user `b1dc16d53d4a407d875c93f3565e414b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:31.595574 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0e23bd62-8c58-4318-aac0-82c346d8a3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:31.663569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1219/2424] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:31.668103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718adfe-176d-4e61-9baf-feededf484fe admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.745875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c718adfe-176d-4e61-9baf-feededf484fe admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.745875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88d701-391c-41b9-8352-059d64b80676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.745875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88d701-391c-41b9-8352-059d64b80676 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.745875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88d701-391c-41b9-8352-059d64b80676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.746263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88d701-391c-41b9-8352-059d64b80676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.746263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88d701-391c-41b9-8352-059d64b80676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.746263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88d701-391c-41b9-8352-059d64b80676 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.774221 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1206/2425] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:31 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:31.778755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23bd62-8c58-4318-aac0-82c346d8a3e2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.790062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23bd62-8c58-4318-aac0-82c346d8a3e2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.824937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:31.851445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849ed01-618e-4417-bbb6-54b27af8424c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.852066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849ed01-618e-4417-bbb6-54b27af8424c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.852338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849ed01-618e-4417-bbb6-54b27af8424c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.852604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849ed01-618e-4417-bbb6-54b27af8424c admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.853225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849ed01-618e-4417-bbb6-54b27af8424c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.855192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849ed01-618e-4417-bbb6-54b27af8424c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.879786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1220/2426] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:31 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:31.883706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88d701-391c-41b9-8352-059d64b80676 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.894902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88d701-391c-41b9-8352-059d64b80676 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.903323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed64e7a-a919-4bca-9f42-6e12c23b0ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.904071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed64e7a-a919-4bca-9f42-6e12c23b0ed9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.904449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed64e7a-a919-4bca-9f42-6e12c23b0ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.904916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed64e7a-a919-4bca-9f42-6e12c23b0ed9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.905766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed64e7a-a919-4bca-9f42-6e12c23b0ed9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.907445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed64e7a-a919-4bca-9f42-6e12c23b0ed9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.924974 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1207/2427] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:31.932162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849ed01-618e-4417-bbb6-54b27af8424c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.942376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849ed01-618e-4417-bbb6-54b27af8424c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:31.947137 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536a59d-d57b-4932-b05e-505153d07de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:31.947859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536a59d-d57b-4932-b05e-505153d07de2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:31.948145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536a59d-d57b-4932-b05e-505153d07de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:31.948413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9536a59d-d57b-4932-b05e-505153d07de2 admin admin] PATH_INFO: `/v3/projects/8a4b5a2f09f84e4d8aa46546711de97e/users/42f195c5739e4147b241eec29809381e/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:31.965907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536a59d-d57b-4932-b05e-505153d07de2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4b5a2f09f84e4d8aa46546711de97e, user_id=42f195c5739e4147b241eec29809381e, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:31.968551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9536a59d-d57b-4932-b05e-505153d07de2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:31.983216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1221/2428] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:31 2022] PUT /identity/v3/projects/8a4b5a2f09f84e4d8aa46546711de97e/users/42f195c5739e4147b241eec29809381e/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:31.987237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed64e7a-a919-4bca-9f42-6e12c23b0ed9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:31.998862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed64e7a-a919-4bca-9f42-6e12c23b0ed9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:32.027264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:32.072533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37945ccf-655a-402b-b921-5522feed83d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:32.073279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945ccf-655a-402b-b921-5522feed83d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:32.073619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945ccf-655a-402b-b921-5522feed83d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:32.074029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37945ccf-655a-402b-b921-5522feed83d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:32.074807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37945ccf-655a-402b-b921-5522feed83d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:32.076806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37945ccf-655a-402b-b921-5522feed83d1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:32.092019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1208/2429] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:31 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:32.096938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536a59d-d57b-4932-b05e-505153d07de2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:32.120064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9536a59d-d57b-4932-b05e-505153d07de2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:32.127041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:32.127861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:32.128203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:32.128537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] PATH_INFO: `/v3/projects/8a4b5a2f09f84e4d8aa46546711de97e/users/42f195c5739e4147b241eec29809381e/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:32.130752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a4b5a2f09f84e4d8aa46546711de97e, user_id=42f195c5739e4147b241eec29809381e, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:32.133525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:32.145707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1222/2430] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:32 2022] PUT /identity/v3/projects/8a4b5a2f09f84e4d8aa46546711de97e/users/42f195c5739e4147b241eec29809381e/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:32.156050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963deba-9739-49b4-b5a8-880ca9093b6a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:32.156565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963deba-9739-49b4-b5a8-880ca9093b6a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:32.156680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6963deba-9739-49b4-b5a8-880ca9093b6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:32.190203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-6963deba-9739-49b4-b5a8-880ca9093b6a None None] MFA Rules not processed for user `42f195c5739e4147b241eec29809381e`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:32.191541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6963deba-9739-49b4-b5a8-880ca9093b6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:32.266192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1209/2431] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:32.279129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:32.281354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:32.344008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226997cf-e77e-4e54-82da-044f2e7bc92a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:32.348136 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b396af7f-c51a-4aaa-aac2-047dc50f68ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:32.348806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396af7f-c51a-4aaa-aac2-047dc50f68ea service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:32.349075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396af7f-c51a-4aaa-aac2-047dc50f68ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:32.349342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b396af7f-c51a-4aaa-aac2-047dc50f68ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:32.358646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396af7f-c51a-4aaa-aac2-047dc50f68ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:32.360098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b396af7f-c51a-4aaa-aac2-047dc50f68ea service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:32.369541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1223/2432] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:33.038056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963deba-9739-49b4-b5a8-880ca9093b6a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:33.105141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6963deba-9739-49b4-b5a8-880ca9093b6a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:33.109807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:33.110449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:33.110720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:33.110985 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] PATH_INFO: `/v3/users/b1dc16d53d4a407d875c93f3565e414b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:33.114305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1dc16d53d4a407d875c93f3565e414b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:33.114305 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:33.193388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1dc16d53d4a407d875c93f3565e414b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.201449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:33.208872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1dc16d53d4a407d875c93f3565e414b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.211044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1dc16d53d4a407d875c93f3565e414b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.223971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1dc16d53d4a407d875c93f3565e414b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.243840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1dc16d53d4a407d875c93f3565e414b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.254425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1210/2433] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:33 2022] DELETE /identity/v3/users/b1dc16d53d4a407d875c93f3565e414b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:33.259391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2480-f9b1-4a71-8f94-54503e03d6bb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:33.259825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2480-f9b1-4a71-8f94-54503e03d6bb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:33.260243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cc2480-f9b1-4a71-8f94-54503e03d6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:33.286058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-93cc2480-f9b1-4a71-8f94-54503e03d6bb None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:33.287171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-93cc2480-f9b1-4a71-8f94-54503e03d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:33.365110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1224/2434] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:33.375858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:33.377394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:33.441087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931bac04-8e02-40cb-b7e2-8d2110a1166a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:33.445061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4ca64-012e-4234-a4ae-db11e11c4c4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:33.445740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4ca64-012e-4234-a4ae-db11e11c4c4c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:33.446010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4ca64-012e-4234-a4ae-db11e11c4c4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:33.446346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b4ca64-012e-4234-a4ae-db11e11c4c4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:33.456890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4ca64-012e-4234-a4ae-db11e11c4c4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:33.458472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b4ca64-012e-4234-a4ae-db11e11c4c4c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:33.468783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1211/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:33.787774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2480-f9b1-4a71-8f94-54503e03d6bb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:33.789051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-93cc2480-f9b1-4a71-8f94-54503e03d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:33.809084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cc2480-f9b1-4a71-8f94-54503e03d6bb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:33.826770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:33.826770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:33.826770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:33.829820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] PATH_INFO: `/v3/projects/b411fbf1d27b4e7f894845ab950825a7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:33.832849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b411fbf1d27b4e7f894845ab950825a7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:33.834925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:33.886742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:33.905234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b411fbf1d27b4e7f894845ab950825a7', 'request_id': 'req-924ee1be-18b5-4194-a863-5b6f6d62ea24'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.911046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b411fbf1d27b4e7f894845ab950825a7', 'request_id': 'req-924ee1be-18b5-4194-a863-5b6f6d62ea24'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.918454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b411fbf1d27b4e7f894845ab950825a7', 'request_id': 'req-924ee1be-18b5-4194-a863-5b6f6d62ea24'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.932866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] The token cache is being invalidate because project b411fbf1d27b4e7f894845ab950825a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:33.933794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.934634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:33.936647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1225/2436] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:33 2022] DELETE /identity/v3/projects/b411fbf1d27b4e7f894845ab950825a7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:33.940916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4ca64-012e-4234-a4ae-db11e11c4c4c service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:33.952430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b4ca64-012e-4234-a4ae-db11e11c4c4c service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:34.016041 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:34.016764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:34.017034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:34.017300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] PATH_INFO: `/v3/users/42f195c5739e4147b241eec29809381e` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:34.018459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42f195c5739e4147b241eec29809381e)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:34.020488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:34.085357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42f195c5739e4147b241eec29809381e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.097831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42f195c5739e4147b241eec29809381e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.098578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42f195c5739e4147b241eec29809381e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.107547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42f195c5739e4147b241eec29809381e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.118348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:34.127593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-fbc1d497-c001-4c26-a82c-8e4d1640cf11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42f195c5739e4147b241eec29809381e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.136136 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1212/2437] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:33 2022] DELETE /identity/v3/users/42f195c5739e4147b241eec29809381e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:34.463288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:34.465170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:34.485415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ee1be-18b5-4194-a863-5b6f6d62ea24 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:34.567835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:34.568613 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:34.570107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:34.570498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] PATH_INFO: `/v3/projects/8a4b5a2f09f84e4d8aa46546711de97e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:34.573595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a4b5a2f09f84e4d8aa46546711de97e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:34.575718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:34.634665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a4b5a2f09f84e4d8aa46546711de97e', 'request_id': 'req-521cee0d-4dfc-4f98-9667-3591e0ade1b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.640429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a4b5a2f09f84e4d8aa46546711de97e', 'request_id': 'req-521cee0d-4dfc-4f98-9667-3591e0ade1b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.647019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a4b5a2f09f84e4d8aa46546711de97e', 'request_id': 'req-521cee0d-4dfc-4f98-9667-3591e0ade1b7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.669648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] The token cache is being invalidate because project 8a4b5a2f09f84e4d8aa46546711de97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:34.670016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.670745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-521cee0d-4dfc-4f98-9667-3591e0ade1b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:34.672524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1226/2438] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:34 2022] DELETE /identity/v3/projects/8a4b5a2f09f84e4d8aa46546711de97e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:34.679087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f7d3-0b1a-44de-b6ce-104d772f7be0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:34.679588 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f7d3-0b1a-44de-b6ce-104d772f7be0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:34.679692 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f9f7d3-0b1a-44de-b6ce-104d772f7be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:34.700110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a9f9f7d3-0b1a-44de-b6ce-104d772f7be0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:34.701403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a9f9f7d3-0b1a-44de-b6ce-104d772f7be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:34.763458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1213/2439] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:34.771654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d8289-f6c2-44ff-9a5b-96ddd29438f1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:34.771914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d8289-f6c2-44ff-9a5b-96ddd29438f1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:34.772184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458d8289-f6c2-44ff-9a5b-96ddd29438f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:34.792837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-458d8289-f6c2-44ff-9a5b-96ddd29438f1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:34.794096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-458d8289-f6c2-44ff-9a5b-96ddd29438f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:34.802759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1227/2440] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:34.807126 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f7d3-0b1a-44de-b6ce-104d772f7be0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:34.817346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f9f7d3-0b1a-44de-b6ce-104d772f7be0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:34.822268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:34.822938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:34.823238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:34.823515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:34.824164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:34.824563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:34.826880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:34.846013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1214/2441] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:43:34.847626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d8289-f6c2-44ff-9a5b-96ddd29438f1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:34.860718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458d8289-f6c2-44ff-9a5b-96ddd29438f1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:34.867702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffa6ff-f5f4-4d0b-9418-8e1391c99f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:34.868580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffa6ff-f5f4-4d0b-9418-8e1391c99f6c admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:34.869014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffa6ff-f5f4-4d0b-9418-8e1391c99f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:34.870036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cffa6ff-f5f4-4d0b-9418-8e1391c99f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:34.870942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffa6ff-f5f4-4d0b-9418-8e1391c99f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:34.875532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cffa6ff-f5f4-4d0b-9418-8e1391c99f6c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:34.914673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1228/2442] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:34 2022] POST /identity/v3/projects => generated 375 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:34.918972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:34.934760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5217232-71eb-4562-bded-6238196fc48a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.025162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4eb1f-82ad-4a45-b66e-c2e665c55178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.025842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4eb1f-82ad-4a45-b66e-c2e665c55178 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.026118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4eb1f-82ad-4a45-b66e-c2e665c55178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.026387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c4eb1f-82ad-4a45-b66e-c2e665c55178 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.027020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4eb1f-82ad-4a45-b66e-c2e665c55178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.029475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c4eb1f-82ad-4a45-b66e-c2e665c55178 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.062578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1215/2443] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:34 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:35.063849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffa6ff-f5f4-4d0b-9418-8e1391c99f6c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.074596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cffa6ff-f5f4-4d0b-9418-8e1391c99f6c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.083510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45e21d-2dfb-463c-b342-117de824e56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.084094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45e21d-2dfb-463c-b342-117de824e56c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.084374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45e21d-2dfb-463c-b342-117de824e56c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.084647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45e21d-2dfb-463c-b342-117de824e56c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.085301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45e21d-2dfb-463c-b342-117de824e56c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.086954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45e21d-2dfb-463c-b342-117de824e56c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.101680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1229/2444] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:35.105969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4eb1f-82ad-4a45-b66e-c2e665c55178 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.127038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c4eb1f-82ad-4a45-b66e-c2e665c55178 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.132539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943bb6c8-de64-489e-ad65-1cb481997217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.133176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb6c8-de64-489e-ad65-1cb481997217 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.133459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb6c8-de64-489e-ad65-1cb481997217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.133758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943bb6c8-de64-489e-ad65-1cb481997217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.134419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bb6c8-de64-489e-ad65-1cb481997217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.136490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943bb6c8-de64-489e-ad65-1cb481997217 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.154833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1216/2445] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:35.159109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45e21d-2dfb-463c-b342-117de824e56c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.169900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45e21d-2dfb-463c-b342-117de824e56c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.175184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116fbba-2ce9-4b68-9338-01b514c7b625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.175860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116fbba-2ce9-4b68-9338-01b514c7b625 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.176141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116fbba-2ce9-4b68-9338-01b514c7b625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.176408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0116fbba-2ce9-4b68-9338-01b514c7b625 admin admin] PATH_INFO: `/v3/projects/a7fdcdccd1ad49aca87bd615f59eb280/users/65d083ec93704d9dbd897140cc1f25ee/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.178983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:35.195473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116fbba-2ce9-4b68-9338-01b514c7b625 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7fdcdccd1ad49aca87bd615f59eb280, user_id=65d083ec93704d9dbd897140cc1f25ee, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.202340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0116fbba-2ce9-4b68-9338-01b514c7b625 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.223087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1230/2446] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:35 2022] PUT /identity/v3/projects/a7fdcdccd1ad49aca87bd615f59eb280/users/65d083ec93704d9dbd897140cc1f25ee/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:35.234641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746539-019a-450b-9e5c-ef86f0ca40f4 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.235213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746539-019a-450b-9e5c-ef86f0ca40f4 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.235483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f746539-019a-450b-9e5c-ef86f0ca40f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.270241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1f746539-019a-450b-9e5c-ef86f0ca40f4 None None] MFA Rules not processed for user `65d083ec93704d9dbd897140cc1f25ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:35.271821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1f746539-019a-450b-9e5c-ef86f0ca40f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:35.347887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:35.375405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1217/2447] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:35.379782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116fbba-2ce9-4b68-9338-01b514c7b625 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.447484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0116fbba-2ce9-4b68-9338-01b514c7b625 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.452857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.453393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.453776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.454106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.455703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.456879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.490811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1231/2448] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:35 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:35.492000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f746539-019a-450b-9e5c-ef86f0ca40f4 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.502725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f746539-019a-450b-9e5c-ef86f0ca40f4 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.531895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.534200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:35.571473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b127e55-98bc-4bdc-88d9-4268ba21699c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.572010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127e55-98bc-4bdc-88d9-4268ba21699c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.572284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127e55-98bc-4bdc-88d9-4268ba21699c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.572549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b127e55-98bc-4bdc-88d9-4268ba21699c admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.573349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b127e55-98bc-4bdc-88d9-4268ba21699c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.575315 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b127e55-98bc-4bdc-88d9-4268ba21699c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.601798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1218/2449] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:35 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:35.604320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b127e55-98bc-4bdc-88d9-4268ba21699c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.608414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e248319f-7bf1-441d-ac24-364d08366c3a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.614350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.614990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.615439 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.615704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] PATH_INFO: `/v3/users/8b6cdfa4ef794d028b9e0a012c0b729f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.616510 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b127e55-98bc-4bdc-88d9-4268ba21699c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.617151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6cdfa4ef794d028b9e0a012c0b729f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.619106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.622169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6a7eb-f500-40d5-9e47-420283a31fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.622858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6a7eb-f500-40d5-9e47-420283a31fec admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.623208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6a7eb-f500-40d5-9e47-420283a31fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.623682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c6a7eb-f500-40d5-9e47-420283a31fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.624392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6a7eb-f500-40d5-9e47-420283a31fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.626118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c6a7eb-f500-40d5-9e47-420283a31fec admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.640668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1219/2450] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:35.644991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6a7eb-f500-40d5-9e47-420283a31fec admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.659220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c6a7eb-f500-40d5-9e47-420283a31fec admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.664840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433830bb-c00f-495d-b3f1-10d514e29aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.665493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433830bb-c00f-495d-b3f1-10d514e29aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.665825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433830bb-c00f-495d-b3f1-10d514e29aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.666133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433830bb-c00f-495d-b3f1-10d514e29aee admin admin] PATH_INFO: `/v3/projects/e4f864dc19bd48e3a976495c9f5f2fa8/users/ff0a143c17fd46cb8246f99e091993e2/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.693336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433830bb-c00f-495d-b3f1-10d514e29aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f864dc19bd48e3a976495c9f5f2fa8, user_id=ff0a143c17fd46cb8246f99e091993e2, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.699331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433830bb-c00f-495d-b3f1-10d514e29aee admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.734759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1220/2451] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:35 2022] PUT /identity/v3/projects/e4f864dc19bd48e3a976495c9f5f2fa8/users/ff0a143c17fd46cb8246f99e091993e2/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:35.736438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433830bb-c00f-495d-b3f1-10d514e29aee admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.818106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433830bb-c00f-495d-b3f1-10d514e29aee admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.818621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6cdfa4ef794d028b9e0a012c0b729f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:35.823180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.823909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.824219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.824514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.825169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.827050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.829729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6cdfa4ef794d028b9e0a012c0b729f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:35.830828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6cdfa4ef794d028b9e0a012c0b729f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:35.841695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6cdfa4ef794d028b9e0a012c0b729f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:35.847571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1221/2452] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:35.849163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:35.852638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:35.855831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:35.867977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a494708b-9cbf-4136-9dbe-fe544f84fa00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6cdfa4ef794d028b9e0a012c0b729f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:35.878067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1232/2453] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:35 2022] DELETE /identity/v3/users/8b6cdfa4ef794d028b9e0a012c0b729f => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:35.878736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a58f38-830c-4271-8f0a-80b316db22c4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:35.883630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578ab41-0910-436a-9a32-361362d099eb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.884235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578ab41-0910-436a-9a32-361362d099eb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.884988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4578ab41-0910-436a-9a32-361362d099eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.913287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4578ab41-0910-436a-9a32-361362d099eb None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:35.923136 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4578ab41-0910-436a-9a32-361362d099eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:35.926119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:35.957038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0b1ec-6dd1-4f7d-b6bd-01a782f325cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:35.957895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b1ec-6dd1-4f7d-b6bd-01a782f325cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.958254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b1ec-6dd1-4f7d-b6bd-01a782f325cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:35.958526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0b1ec-6dd1-4f7d-b6bd-01a782f325cf admin admin] PATH_INFO: `/v3/projects/e4f864dc19bd48e3a976495c9f5f2fa8/users/ff0a143c17fd46cb8246f99e091993e2/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:35.960535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0b1ec-6dd1-4f7d-b6bd-01a782f325cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f864dc19bd48e3a976495c9f5f2fa8, user_id=ff0a143c17fd46cb8246f99e091993e2, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:35.964698 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0b1ec-6dd1-4f7d-b6bd-01a782f325cf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:35.983942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1222/2454] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:35 2022] PUT /identity/v3/projects/e4f864dc19bd48e3a976495c9f5f2fa8/users/ff0a143c17fd46cb8246f99e091993e2/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:35.999547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0123f-9e62-4ab7-a7aa-797ea73bc3fd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:35.999922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0123f-9e62-4ab7-a7aa-797ea73bc3fd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:36.000306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a0123f-9e62-4ab7-a7aa-797ea73bc3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:36.021932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1233/2455] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:36.030065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578ab41-0910-436a-9a32-361362d099eb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:36.031762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4578ab41-0910-436a-9a32-361362d099eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:36.034222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-24a0123f-9e62-4ab7-a7aa-797ea73bc3fd None None] MFA Rules not processed for user `ff0a143c17fd46cb8246f99e091993e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:36.035829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-24a0123f-9e62-4ab7-a7aa-797ea73bc3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:36.128179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4578ab41-0910-436a-9a32-361362d099eb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:36.147333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1223/2456] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:35 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:36.148684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0123f-9e62-4ab7-a7aa-797ea73bc3fd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:36.151485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-24a0123f-9e62-4ab7-a7aa-797ea73bc3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:36.233308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:36.237925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:36.237925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:36.237925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:36.284475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a0123f-9e62-4ab7-a7aa-797ea73bc3fd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:36.294099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b657209-6030-4634-b21b-4aa482710b0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:36.294099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b657209-6030-4634-b21b-4aa482710b0c service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:36.294387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b657209-6030-4634-b21b-4aa482710b0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:36.294464 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b657209-6030-4634-b21b-4aa482710b0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:36.296406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4b657209-6030-4634-b21b-4aa482710b0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:36.313622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:36.335199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:36.337133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:36.356279 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1234/2457] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:36.378536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b657209-6030-4634-b21b-4aa482710b0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:36.380520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b657209-6030-4634-b21b-4aa482710b0c service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:36.395097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1224/2458] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:36.717440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:36.719353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:36.754801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d263cc9-445a-45bb-ab77-08f4fa079dd0 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:36.760827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:36.761602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:36.761966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:36.762243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] PATH_INFO: `/v3/projects/fe2a18b0ac7d4135828d5cec8cee04e7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:36.763421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2a18b0ac7d4135828d5cec8cee04e7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:36.765553 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:36.831945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2a18b0ac7d4135828d5cec8cee04e7', 'request_id': 'req-45a8dbcb-4275-4fae-ac18-9597618a0551'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:36.839411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2a18b0ac7d4135828d5cec8cee04e7', 'request_id': 'req-45a8dbcb-4275-4fae-ac18-9597618a0551'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:36.848647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2a18b0ac7d4135828d5cec8cee04e7', 'request_id': 'req-45a8dbcb-4275-4fae-ac18-9597618a0551'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:36.865936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] The token cache is being invalidate because project fe2a18b0ac7d4135828d5cec8cee04e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:36.866507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:36.867542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-45a8dbcb-4275-4fae-ac18-9597618a0551 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:36.869798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1235/2459] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:36 2022] DELETE /identity/v3/projects/fe2a18b0ac7d4135828d5cec8cee04e7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:36.875077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f5ac3-c12b-4dc9-b61f-ac10cff9751f None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:36.875580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f5ac3-c12b-4dc9-b61f-ac10cff9751f None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:36.875675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f5ac3-c12b-4dc9-b61f-ac10cff9751f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:36.899579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d40f5ac3-c12b-4dc9-b61f-ac10cff9751f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:36.900792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d40f5ac3-c12b-4dc9-b61f-ac10cff9751f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:36.905109 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:36.962275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1225/2460] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:36.971491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48010d-2244-4ab4-9373-10067b13fcc1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:36.971899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48010d-2244-4ab4-9373-10067b13fcc1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:36.972298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c48010d-2244-4ab4-9373-10067b13fcc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.000612 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c48010d-2244-4ab4-9373-10067b13fcc1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:37.002691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c48010d-2244-4ab4-9373-10067b13fcc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:37.015332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1236/2461] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:37.021164 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f5ac3-c12b-4dc9-b61f-ac10cff9751f None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:37.037957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f5ac3-c12b-4dc9-b61f-ac10cff9751f None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:37.043689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:37.044390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:37.044738 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:37.045045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.045928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:37.046232 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:37.047927 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:37.063326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1226/2462] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:43:37.078881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48010d-2244-4ab4-9373-10067b13fcc1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:37.089972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c48010d-2244-4ab4-9373-10067b13fcc1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:37.103170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413aa68-be3d-4fe6-8078-cfa4ef98ecf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:37.103798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413aa68-be3d-4fe6-8078-cfa4ef98ecf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:37.104073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413aa68-be3d-4fe6-8078-cfa4ef98ecf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:37.104340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413aa68-be3d-4fe6-8078-cfa4ef98ecf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.104974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413aa68-be3d-4fe6-8078-cfa4ef98ecf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:37.106922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6413aa68-be3d-4fe6-8078-cfa4ef98ecf0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:37.138710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1237/2463] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:37 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:37.142913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:37.156535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e9292-1b49-4365-b797-8fd4aea4ec55 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:37.223167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7b02-b257-4e27-9738-c98b90d3d83d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:37.223879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7b02-b257-4e27-9738-c98b90d3d83d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:37.224155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7b02-b257-4e27-9738-c98b90d3d83d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:37.224422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f7b02-b257-4e27-9738-c98b90d3d83d admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.225264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f7b02-b257-4e27-9738-c98b90d3d83d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:37.227120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f7b02-b257-4e27-9738-c98b90d3d83d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:37.250698 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1227/2464] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:37 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:37.255365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413aa68-be3d-4fe6-8078-cfa4ef98ecf0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:37.266479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413aa68-be3d-4fe6-8078-cfa4ef98ecf0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:37.271225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0231991-c9d9-470b-bbcd-69c920342344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:37.271840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231991-c9d9-470b-bbcd-69c920342344 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:37.272176 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231991-c9d9-470b-bbcd-69c920342344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:37.272389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0231991-c9d9-470b-bbcd-69c920342344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.273072 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0231991-c9d9-470b-bbcd-69c920342344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:37.274715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0231991-c9d9-470b-bbcd-69c920342344 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:37.285316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1238/2465] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:37.289679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7b02-b257-4e27-9738-c98b90d3d83d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:37.305586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f7b02-b257-4e27-9738-c98b90d3d83d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:37.311309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec63b9a-df3e-4b44-b97d-9294473cd708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:37.311937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec63b9a-df3e-4b44-b97d-9294473cd708 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:37.312176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec63b9a-df3e-4b44-b97d-9294473cd708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:37.312468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec63b9a-df3e-4b44-b97d-9294473cd708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.313122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec63b9a-df3e-4b44-b97d-9294473cd708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:37.314772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec63b9a-df3e-4b44-b97d-9294473cd708 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:37.329690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1228/2466] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:37.331896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0231991-c9d9-470b-bbcd-69c920342344 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:37.343113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0231991-c9d9-470b-bbcd-69c920342344 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:37.348705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:37.349468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:37.349801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:37.350073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] PATH_INFO: `/v3/projects/93588cc1d2704a73ae5a76dcc6fc44f1/users/dea49d304629415895813b6ab7f9c9db/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.368290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93588cc1d2704a73ae5a76dcc6fc44f1, user_id=dea49d304629415895813b6ab7f9c9db, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:37.371189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:37.387804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1239/2467] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:37 2022] PUT /identity/v3/projects/93588cc1d2704a73ae5a76dcc6fc44f1/users/dea49d304629415895813b6ab7f9c9db/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:37.394146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb25bd-24fc-4764-bad1-379933533b1e None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:37.394425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb25bd-24fc-4764-bad1-379933533b1e None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:37.394697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb25bd-24fc-4764-bad1-379933533b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.413505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e5fb25bd-24fc-4764-bad1-379933533b1e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:37.414768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e5fb25bd-24fc-4764-bad1-379933533b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:37.478264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1229/2468] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:37.488252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:37.490092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:37.558940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d874182-6be4-4fcc-9f41-2fda8bc761d3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:37.563106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:37.563770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:37.564046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:37.564315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:37.574408 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:37.575859 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:37.585766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1240/2469] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:39.127467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb25bd-24fc-4764-bad1-379933533b1e None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:39.130124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e5fb25bd-24fc-4764-bad1-379933533b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:39.144044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb25bd-24fc-4764-bad1-379933533b1e None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:39.149902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:39.150570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:39.150845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:39.151122 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] PATH_INFO: `/v3/users/65d083ec93704d9dbd897140cc1f25ee` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:39.152461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65d083ec93704d9dbd897140cc1f25ee)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:39.154469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:39.168781 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:39.234454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:39.237406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:39.251282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:39.262022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bd7ab-efcb-4cf0-a78e-1ad9693a7e6c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:39.271937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a46612-fe30-4e1c-89a3-0af94307fa52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:39.272824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a46612-fe30-4e1c-89a3-0af94307fa52 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:39.273121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a46612-fe30-4e1c-89a3-0af94307fa52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:39.273431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a46612-fe30-4e1c-89a3-0af94307fa52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:39.275951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-63a46612-fe30-4e1c-89a3-0af94307fa52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:39.284236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65d083ec93704d9dbd897140cc1f25ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:39.293038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a46612-fe30-4e1c-89a3-0af94307fa52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:39.294105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65d083ec93704d9dbd897140cc1f25ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:39.294875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a46612-fe30-4e1c-89a3-0af94307fa52 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:39.295093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65d083ec93704d9dbd897140cc1f25ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:39.305764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1241/2470] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:39.310836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65d083ec93704d9dbd897140cc1f25ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:39.331230 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65d083ec93704d9dbd897140cc1f25ee'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:39.339068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1230/2471] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:39 2022] DELETE /identity/v3/users/65d083ec93704d9dbd897140cc1f25ee => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:39.343983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1561f-883a-4cea-a248-7b6f092c291c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:39.344265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1561f-883a-4cea-a248-7b6f092c291c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:39.345121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f1561f-883a-4cea-a248-7b6f092c291c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:39.370626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-83f1561f-883a-4cea-a248-7b6f092c291c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:39.371999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83f1561f-883a-4cea-a248-7b6f092c291c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:39.475979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1242/2472] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:39.491277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:39.492747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:39.593112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3670f5-47be-483c-b47b-052b22588527 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:39.598424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:39.599236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:39.599517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:39.599786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:39.610271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1561f-883a-4cea-a248-7b6f092c291c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:39.623472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f1561f-883a-4cea-a248-7b6f092c291c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:39.625921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:39.628335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:39.632103 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:39.633160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:39.633617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:39.634244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:39.636423 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:39.641552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1231/2473] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:39.651772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:39.653259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:39.665359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1243/2474] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:41.153827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:41.162776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:41.188023 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccdf829-e322-4200-9f34-7916aa949dab service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:41.207978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:41.208668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:41.209149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:41.209577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] PATH_INFO: `/v3/projects/a7fdcdccd1ad49aca87bd615f59eb280` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:41.213018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7fdcdccd1ad49aca87bd615f59eb280)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:41.215748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:41.321453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7fdcdccd1ad49aca87bd615f59eb280', 'request_id': 'req-a5bcbee9-1970-427b-9803-b1f6587dbfe9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.328128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7fdcdccd1ad49aca87bd615f59eb280', 'request_id': 'req-a5bcbee9-1970-427b-9803-b1f6587dbfe9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.335734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7fdcdccd1ad49aca87bd615f59eb280', 'request_id': 'req-a5bcbee9-1970-427b-9803-b1f6587dbfe9'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.359793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] The token cache is being invalidate because project a7fdcdccd1ad49aca87bd615f59eb280 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:41.360227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.361000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.365686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1232/2475] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:41 2022] DELETE /identity/v3/projects/a7fdcdccd1ad49aca87bd615f59eb280 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:41.367619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:41.396633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd11d1-5d7c-457f-adad-57b5767cacd9 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:41.570674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:41.571498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:41.571778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:41.572049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] PATH_INFO: `/v3/users/ff0a143c17fd46cb8246f99e091993e2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:41.574930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff0a143c17fd46cb8246f99e091993e2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:41.579914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:41.662876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff0a143c17fd46cb8246f99e091993e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.672848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff0a143c17fd46cb8246f99e091993e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.674741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff0a143c17fd46cb8246f99e091993e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.688942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff0a143c17fd46cb8246f99e091993e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.711622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff0a143c17fd46cb8246f99e091993e2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:41.713614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:41.723046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1244/2476] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:41 2022] DELETE /identity/v3/users/ff0a143c17fd46cb8246f99e091993e2 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:42.008241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:42.010060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:42.048692 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bcbee9-1970-427b-9803-b1f6587dbfe9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:42.062728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:42.140013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:42.140824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:42.141102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:42.141376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] PATH_INFO: `/v3/users/70276ea5068f4bfc85d4216aca67fb1d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:42.142814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70276ea5068f4bfc85d4216aca67fb1d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:42.144947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:42.245105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70276ea5068f4bfc85d4216aca67fb1d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.259344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70276ea5068f4bfc85d4216aca67fb1d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.260177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70276ea5068f4bfc85d4216aca67fb1d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.287766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70276ea5068f4bfc85d4216aca67fb1d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.316006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:42.317848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:42.329083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f44a7719-fea9-4dbf-8994-7e4f64cfdb6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70276ea5068f4bfc85d4216aca67fb1d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.342547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2cb815-f275-474c-9ff9-0fc851b6cb56 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:42.355373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:42.356377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:42.356812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:42.357345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] PATH_INFO: `/v3/projects/e4f864dc19bd48e3a976495c9f5f2fa8` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:42.362708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f864dc19bd48e3a976495c9f5f2fa8)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:42.365703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:42.368512 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1233/2477] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:42 2022] DELETE /identity/v3/users/70276ea5068f4bfc85d4216aca67fb1d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:42.443289 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f864dc19bd48e3a976495c9f5f2fa8', 'request_id': 'req-54aa19b8-8ede-4bd6-b6e1-760911a38e00'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.452879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f864dc19bd48e3a976495c9f5f2fa8', 'request_id': 'req-54aa19b8-8ede-4bd6-b6e1-760911a38e00'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.463187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f864dc19bd48e3a976495c9f5f2fa8', 'request_id': 'req-54aa19b8-8ede-4bd6-b6e1-760911a38e00'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.513384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] The token cache is being invalidate because project e4f864dc19bd48e3a976495c9f5f2fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:42.513955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.517258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-54aa19b8-8ede-4bd6-b6e1-760911a38e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.519141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1245/2478] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:42 2022] DELETE /identity/v3/projects/e4f864dc19bd48e3a976495c9f5f2fa8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:42.526352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119845-daaf-4e79-9949-9b0580507187 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:42.526815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119845-daaf-4e79-9949-9b0580507187 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:42.527130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119845-daaf-4e79-9949-9b0580507187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:42.562682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f3119845-daaf-4e79-9949-9b0580507187 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:42.564119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f3119845-daaf-4e79-9949-9b0580507187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:42.653876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1234/2479] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:42.663509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3467-d11e-4f8b-a2d7-e6805fdcd0b4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:42.663790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3467-d11e-4f8b-a2d7-e6805fdcd0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:42.664062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3467-d11e-4f8b-a2d7-e6805fdcd0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:42.691713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4fbf3467-d11e-4f8b-a2d7-e6805fdcd0b4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:42.693227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4fbf3467-d11e-4f8b-a2d7-e6805fdcd0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:42.706822 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1246/2480] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:42.711155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3119845-daaf-4e79-9949-9b0580507187 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:42.727415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3119845-daaf-4e79-9949-9b0580507187 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:42.743957 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:42.744661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:42.744937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:42.745225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:42.747089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:42.747425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:42.751526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:42.789024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3467-d11e-4f8b-a2d7-e6805fdcd0b4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:42.790834 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4fbf3467-d11e-4f8b-a2d7-e6805fdcd0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:42.797794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1235/2481] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:43:42.801746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:42.815376 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ef794-2877-4640-85c2-141e627d2079 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:42.820999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdfbae-84d7-4335-a28e-960a07bf65be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:42.821541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdfbae-84d7-4335-a28e-960a07bf65be admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:42.821817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdfbae-84d7-4335-a28e-960a07bf65be admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:42.822091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdfbae-84d7-4335-a28e-960a07bf65be admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:42.822755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdfbae-84d7-4335-a28e-960a07bf65be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:42.826964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbdfbae-84d7-4335-a28e-960a07bf65be admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:42.847155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3467-d11e-4f8b-a2d7-e6805fdcd0b4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:42.859538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:42.860361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:42.860711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:42.860951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] PATH_INFO: `/v3/projects/04eff040818b4f2eb4fe7f7e9dbc8460` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:42.862234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=04eff040818b4f2eb4fe7f7e9dbc8460)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:42.864577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:42.870352 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:42.887511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1236/2482] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:42 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:42.888449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdfbae-84d7-4335-a28e-960a07bf65be admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:42.949502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04eff040818b4f2eb4fe7f7e9dbc8460', 'request_id': 'req-8e1c944d-abd9-4e02-84ac-e3713161d5ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.967071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04eff040818b4f2eb4fe7f7e9dbc8460', 'request_id': 'req-8e1c944d-abd9-4e02-84ac-e3713161d5ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:42.992699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdfbae-84d7-4335-a28e-960a07bf65be admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:43.011006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04eff040818b4f2eb4fe7f7e9dbc8460', 'request_id': 'req-8e1c944d-abd9-4e02-84ac-e3713161d5ef'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:43.095618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] The token cache is being invalidate because project 04eff040818b4f2eb4fe7f7e9dbc8460 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:43.095914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:43.096743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:43.102638 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1247/2483] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:42 2022] DELETE /identity/v3/projects/04eff040818b4f2eb4fe7f7e9dbc8460 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:43.103656 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:43.107133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:43.172970 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:43.174204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:43.174570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:43.174959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:43.175750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:43.178814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:43.241697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1237/2484] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:42 2022] POST /identity/v3/users => generated 292 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:43.241697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:43.244190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:43.270739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:43.276031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c944d-abd9-4e02-84ac-e3713161d5ef admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:43.286394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:43.288087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:43.289115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:43.290531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] PATH_INFO: `/v3/users/3be9f4e2773641a695d7e8ab11174c9b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:43.292223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3be9f4e2773641a695d7e8ab11174c9b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:43.296346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:43.317489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7581cfe-e34e-455d-b7c2-9a9553d3abbd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:43.326838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafc6b1-2fcc-45bf-abc0-911555e26659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:43.327785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc6b1-2fcc-45bf-abc0-911555e26659 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:43.327785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc6b1-2fcc-45bf-abc0-911555e26659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:43.328069 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fafc6b1-2fcc-45bf-abc0-911555e26659 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:43.328705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafc6b1-2fcc-45bf-abc0-911555e26659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:43.330403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fafc6b1-2fcc-45bf-abc0-911555e26659 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:43.361756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1238/2485] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:43.366402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafc6b1-2fcc-45bf-abc0-911555e26659 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:43.369955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3be9f4e2773641a695d7e8ab11174c9b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:43.383434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fafc6b1-2fcc-45bf-abc0-911555e26659 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:43.383590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3be9f4e2773641a695d7e8ab11174c9b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:43.384930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3be9f4e2773641a695d7e8ab11174c9b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:43.391476 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:43.392292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:43.392672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:43.393039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:43.400665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3be9f4e2773641a695d7e8ab11174c9b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:43.402176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:43.404636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:43.420128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-660a8d83-5c7f-4e49-921d-13111bed9eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3be9f4e2773641a695d7e8ab11174c9b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:43.422113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1239/2486] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:43.425970 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:43.431622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1248/2487] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:43 2022] DELETE /identity/v3/users/3be9f4e2773641a695d7e8ab11174c9b => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:43.435206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:43.451361 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592ee815-5365-4e77-9fa6-962bae5947f4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:43.580068 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157c755-1b28-44fe-bbd5-fb55f17d4fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:43.581245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157c755-1b28-44fe-bbd5-fb55f17d4fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:43.581592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157c755-1b28-44fe-bbd5-fb55f17d4fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:43.581942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157c755-1b28-44fe-bbd5-fb55f17d4fe8 admin admin] PATH_INFO: `/v3/projects/80a339e545ea4dc9be93617f26c91448/users/32c6de95541c47cbbbb1ebc2c2e724c0/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:43.622154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157c755-1b28-44fe-bbd5-fb55f17d4fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a339e545ea4dc9be93617f26c91448, user_id=32c6de95541c47cbbbb1ebc2c2e724c0, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:43.629720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157c755-1b28-44fe-bbd5-fb55f17d4fe8 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:43.638875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:43.668933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1240/2488] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:43 2022] PUT /identity/v3/projects/80a339e545ea4dc9be93617f26c91448/users/32c6de95541c47cbbbb1ebc2c2e724c0/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:43:43.679484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725bb53-21f0-44c1-a5f7-2b80b223e77d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:43.680037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725bb53-21f0-44c1-a5f7-2b80b223e77d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:43.680037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f725bb53-21f0-44c1-a5f7-2b80b223e77d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:43.747162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f725bb53-21f0-44c1-a5f7-2b80b223e77d None None] MFA Rules not processed for user `32c6de95541c47cbbbb1ebc2c2e724c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:43.751085 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f725bb53-21f0-44c1-a5f7-2b80b223e77d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:43.881191 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1249/2489] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:43.885502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157c755-1b28-44fe-bbd5-fb55f17d4fe8 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:43.985284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157c755-1b28-44fe-bbd5-fb55f17d4fe8 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:43.990000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e38f6a-9fa7-4add-a5b3-f5c76f9c9048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:43.990728 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e38f6a-9fa7-4add-a5b3-f5c76f9c9048 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:43.991062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e38f6a-9fa7-4add-a5b3-f5c76f9c9048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:43.991328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e38f6a-9fa7-4add-a5b3-f5c76f9c9048 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:43.991996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e38f6a-9fa7-4add-a5b3-f5c76f9c9048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:43.993911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e38f6a-9fa7-4add-a5b3-f5c76f9c9048 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.027034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1241/2490] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:43 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:44.030673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725bb53-21f0-44c1-a5f7-2b80b223e77d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.046288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f725bb53-21f0-44c1-a5f7-2b80b223e77d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.132100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade171a-c5cd-42c1-8e9e-42bd67ae5e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.132629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade171a-c5cd-42c1-8e9e-42bd67ae5e8f admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.132905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade171a-c5cd-42c1-8e9e-42bd67ae5e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.133171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade171a-c5cd-42c1-8e9e-42bd67ae5e8f admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.133881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade171a-c5cd-42c1-8e9e-42bd67ae5e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.135814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade171a-c5cd-42c1-8e9e-42bd67ae5e8f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.148825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:44.166401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1250/2491] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:43:44.170848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e38f6a-9fa7-4add-a5b3-f5c76f9c9048 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.183794 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e38f6a-9fa7-4add-a5b3-f5c76f9c9048 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.190410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c964d-9e1c-4cc3-ad11-0fb1d746f015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.191106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c964d-9e1c-4cc3-ad11-0fb1d746f015 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.191455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c964d-9e1c-4cc3-ad11-0fb1d746f015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.191904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0c964d-9e1c-4cc3-ad11-0fb1d746f015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.192626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c964d-9e1c-4cc3-ad11-0fb1d746f015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.194323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0c964d-9e1c-4cc3-ad11-0fb1d746f015 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.212015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1242/2492] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:44.216964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade171a-c5cd-42c1-8e9e-42bd67ae5e8f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.227276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade171a-c5cd-42c1-8e9e-42bd67ae5e8f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.232240 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7972445c-f7e3-4bea-88e5-f87734f45204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.232969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972445c-f7e3-4bea-88e5-f87734f45204 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.233303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972445c-f7e3-4bea-88e5-f87734f45204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.233625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7972445c-f7e3-4bea-88e5-f87734f45204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.234379 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972445c-f7e3-4bea-88e5-f87734f45204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.236049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7972445c-f7e3-4bea-88e5-f87734f45204 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.246294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1251/2493] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:43:44.250704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c964d-9e1c-4cc3-ad11-0fb1d746f015 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.261413 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0c964d-9e1c-4cc3-ad11-0fb1d746f015 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.268434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.268923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.269115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.269393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] PATH_INFO: `/v3/projects/a2392e8568a74658bb3a6a97a7fcc541/users/fd2f7daae20b465ebe9ba0b3a39db49c/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.289908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2392e8568a74658bb3a6a97a7fcc541, user_id=fd2f7daae20b465ebe9ba0b3a39db49c, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.293491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.310784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1243/2494] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:44 2022] PUT /identity/v3/projects/a2392e8568a74658bb3a6a97a7fcc541/users/fd2f7daae20b465ebe9ba0b3a39db49c/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:44.321719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edea0e-328c-46b4-8747-8d4a9aabf2fa None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.322006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edea0e-328c-46b4-8747-8d4a9aabf2fa None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.322976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61edea0e-328c-46b4-8747-8d4a9aabf2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.357225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-61edea0e-328c-46b4-8747-8d4a9aabf2fa None None] MFA Rules not processed for user `fd2f7daae20b465ebe9ba0b3a39db49c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:44.358663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-61edea0e-328c-46b4-8747-8d4a9aabf2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:44.396421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.398775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:44.458498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1252/2495] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:44.477590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7333-2e61-4c9d-b945-f11429fb39b2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.478243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7333-2e61-4c9d-b945-f11429fb39b2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.478821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4be7333-2e61-4c9d-b945-f11429fb39b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.496683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882b395-647d-4173-a1c5-dfb3c01ee67e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.509965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.510887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.511166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.511434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] PATH_INFO: `/v3/projects/a1dde76be40c49539db2c4ed29d0cd04` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.511882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4be7333-2e61-4c9d-b945-f11429fb39b2 None None] MFA Rules not processed for user `dea49d304629415895813b6ab7f9c9db`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:44.512624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1dde76be40c49539db2c4ed29d0cd04)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.513128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4be7333-2e61-4c9d-b945-f11429fb39b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:44.514683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.564529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:44.565912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1dde76be40c49539db2c4ed29d0cd04', 'request_id': 'req-e26c59b7-7e7e-4458-b705-e69711c81742'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:44.573816 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1dde76be40c49539db2c4ed29d0cd04', 'request_id': 'req-e26c59b7-7e7e-4458-b705-e69711c81742'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:44.579284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1253/2496] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:43:44.580698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be7333-2e61-4c9d-b945-f11429fb39b2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.581552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1dde76be40c49539db2c4ed29d0cd04', 'request_id': 'req-e26c59b7-7e7e-4458-b705-e69711c81742'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:44.592285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4be7333-2e61-4c9d-b945-f11429fb39b2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.598060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] The token cache is being invalidate because project a1dde76be40c49539db2c4ed29d0cd04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:43:44.598363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:44.599022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:44.601845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1244/2497] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:43:44 2022] DELETE /identity/v3/projects/a1dde76be40c49539db2c4ed29d0cd04 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:44.602744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.604139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:44.657128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdee8b8-4bf6-4d88-adae-119bc8b36f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.657775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdee8b8-4bf6-4d88-adae-119bc8b36f23 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.658046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdee8b8-4bf6-4d88-adae-119bc8b36f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.658309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdee8b8-4bf6-4d88-adae-119bc8b36f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.658972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdee8b8-4bf6-4d88-adae-119bc8b36f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.660879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdee8b8-4bf6-4d88-adae-119bc8b36f23 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.670410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:44.670883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c59b7-7e7e-4458-b705-e69711c81742 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.676253 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.676879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.677154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.677428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.678097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.680060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.689471 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1254/2498] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:44.690826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4505138-d174-495c-9018-d24bfebb3a37 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.691107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4505138-d174-495c-9018-d24bfebb3a37 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.691371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4505138-d174-495c-9018-d24bfebb3a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.712108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e4505138-d174-495c-9018-d24bfebb3a37 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:44.717548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1245/2499] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/projects => generated 379 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:44.717805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e4505138-d174-495c-9018-d24bfebb3a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:44.717935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.720077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:44.804542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79982e27-a0df-4342-84db-414d2025eb49 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.805183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1255/2500] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:44.806248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4505138-d174-495c-9018-d24bfebb3a37 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.814786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf0bec-d1f8-4d93-8849-f4e44245c4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.815415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf0bec-d1f8-4d93-8849-f4e44245c4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.815686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf0bec-d1f8-4d93-8849-f4e44245c4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.815949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf0bec-d1f8-4d93-8849-f4e44245c4f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.816596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf0bec-d1f8-4d93-8849-f4e44245c4f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.817311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4505138-d174-495c-9018-d24bfebb3a37 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.818568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edf0bec-d1f8-4d93-8849-f4e44245c4f6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.821961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b8be7-120f-4c8d-b8bd-4ad0f6ef2001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.822614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b8be7-120f-4c8d-b8bd-4ad0f6ef2001 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.822893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b8be7-120f-4c8d-b8bd-4ad0f6ef2001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.823160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b8be7-120f-4c8d-b8bd-4ad0f6ef2001 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.823790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b8be7-120f-4c8d-b8bd-4ad0f6ef2001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.825787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b8be7-120f-4c8d-b8bd-4ad0f6ef2001 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.843403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1246/2501] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/users => generated 299 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:44.844898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04daf6c8-a843-4d19-9764-faedbd0b72a5 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.845176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04daf6c8-a843-4d19-9764-faedbd0b72a5 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.845444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04daf6c8-a843-4d19-9764-faedbd0b72a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.853467 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1256/2502] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/users => generated 296 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:44.855832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b8be7-120f-4c8d-b8bd-4ad0f6ef2001 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.866527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b8be7-120f-4c8d-b8bd-4ad0f6ef2001 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.867107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-04daf6c8-a843-4d19-9764-faedbd0b72a5 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:44.868349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04daf6c8-a843-4d19-9764-faedbd0b72a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:44.871585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d292a5-cc77-43bb-8620-063a0b6bce2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.872280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d292a5-cc77-43bb-8620-063a0b6bce2e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.872628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d292a5-cc77-43bb-8620-063a0b6bce2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.873114 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d292a5-cc77-43bb-8620-063a0b6bce2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.874764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d292a5-cc77-43bb-8620-063a0b6bce2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.876556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d292a5-cc77-43bb-8620-063a0b6bce2e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.877890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1247/2503] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:44.878186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04daf6c8-a843-4d19-9764-faedbd0b72a5 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.891192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04daf6c8-a843-4d19-9764-faedbd0b72a5 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.892270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1257/2504] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:44.893126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d292a5-cc77-43bb-8620-063a0b6bce2e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.899829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132eb3-94e3-45e2-89de-e1aade2f583c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.900663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132eb3-94e3-45e2-89de-e1aade2f583c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.901081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132eb3-94e3-45e2-89de-e1aade2f583c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.901345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8132eb3-94e3-45e2-89de-e1aade2f583c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.902098 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8132eb3-94e3-45e2-89de-e1aade2f583c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.903764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8132eb3-94e3-45e2-89de-e1aade2f583c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.907427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d292a5-cc77-43bb-8620-063a0b6bce2e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.912126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.912757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.913030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.913295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.913968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:43:44.914264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.915921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1248/2505] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:44.916053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.917024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132eb3-94e3-45e2-89de-e1aade2f583c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.928340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8132eb3-94e3-45e2-89de-e1aade2f583c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.930997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1258/2506] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:43:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:43:44.931664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.933603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8503c-4087-46ee-a985-7e26a6043714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.934452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8503c-4087-46ee-a985-7e26a6043714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.934814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8503c-4087-46ee-a985-7e26a6043714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.935152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd8503c-4087-46ee-a985-7e26a6043714 admin admin] PATH_INFO: `/v3/projects/b01d8810c1db4ccf8c89975ce0524785/users/038efbc1a3f04f378194dca84bdc1490/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.943454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a51374-8278-49e3-8397-928a6fc95c90 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:44.948017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5817f9e-4e87-4b77-8b2b-33d95204c5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:44.948664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5817f9e-4e87-4b77-8b2b-33d95204c5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:44.948931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5817f9e-4e87-4b77-8b2b-33d95204c5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:44.949201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5817f9e-4e87-4b77-8b2b-33d95204c5a6 admin admin] PATH_INFO: `/v3/projects/dc21568194bb4facbc9c971671462aa3/users/e7e9578eec534990ba636c779ca93259/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:44.950270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8503c-4087-46ee-a985-7e26a6043714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01d8810c1db4ccf8c89975ce0524785, user_id=038efbc1a3f04f378194dca84bdc1490, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.953025 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd8503c-4087-46ee-a985-7e26a6043714 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.967067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5817f9e-4e87-4b77-8b2b-33d95204c5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc21568194bb4facbc9c971671462aa3, user_id=e7e9578eec534990ba636c779ca93259, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:44.969696 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5817f9e-4e87-4b77-8b2b-33d95204c5a6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:44.970064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1249/2507] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:44 2022] PUT /identity/v3/projects/b01d8810c1db4ccf8c89975ce0524785/users/038efbc1a3f04f378194dca84bdc1490/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:44.971022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8503c-4087-46ee-a985-7e26a6043714 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:44.984812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=91,keystone:DELETE=3,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:44.987008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1259/2508] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:44 2022] PUT /identity/v3/projects/dc21568194bb4facbc9c971671462aa3/users/e7e9578eec534990ba636c779ca93259/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:44.987850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5817f9e-4e87-4b77-8b2b-33d95204c5a6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.058503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd8503c-4087-46ee-a985-7e26a6043714 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.075929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5817f9e-4e87-4b77-8b2b-33d95204c5a6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.090897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13b250-b05f-4a2d-871f-aace27fc9d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.092047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b250-b05f-4a2d-871f-aace27fc9d30 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.092738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b250-b05f-4a2d-871f-aace27fc9d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.093346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13b250-b05f-4a2d-871f-aace27fc9d30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.094446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13b250-b05f-4a2d-871f-aace27fc9d30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.096697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13b250-b05f-4a2d-871f-aace27fc9d30 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.111898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1260/2509] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:44 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:45.113145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13b250-b05f-4a2d-871f-aace27fc9d30 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.126419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13b250-b05f-4a2d-871f-aace27fc9d30 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.131564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7087a-3668-4b26-a985-191f78feb2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.132177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7087a-3668-4b26-a985-191f78feb2ca admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.132444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7087a-3668-4b26-a985-191f78feb2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.132709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c7087a-3668-4b26-a985-191f78feb2ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.133348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7087a-3668-4b26-a985-191f78feb2ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.134992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c7087a-3668-4b26-a985-191f78feb2ca admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.135921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be168e55-1070-44c1-9be8-22227bd5dcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.136834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be168e55-1070-44c1-9be8-22227bd5dcac admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.137935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be168e55-1070-44c1-9be8-22227bd5dcac admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.138438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be168e55-1070-44c1-9be8-22227bd5dcac admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.139575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be168e55-1070-44c1-9be8-22227bd5dcac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.141855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be168e55-1070-44c1-9be8-22227bd5dcac admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.145461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1261/2510] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:45.146290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7087a-3668-4b26-a985-191f78feb2ca admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.156268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c7087a-3668-4b26-a985-191f78feb2ca admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.161777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7fe46-a94c-402e-80ac-d050d1f96daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.162486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7fe46-a94c-402e-80ac-d050d1f96daf admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.162774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7fe46-a94c-402e-80ac-d050d1f96daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.163045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7fe46-a94c-402e-80ac-d050d1f96daf admin admin] PATH_INFO: `/v3/projects/b01d8810c1db4ccf8c89975ce0524785/users/038efbc1a3f04f378194dca84bdc1490/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.165268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7fe46-a94c-402e-80ac-d050d1f96daf admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01d8810c1db4ccf8c89975ce0524785, user_id=038efbc1a3f04f378194dca84bdc1490, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.167878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb7fe46-a94c-402e-80ac-d050d1f96daf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.171376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:45.181908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1262/2511] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:45 2022] PUT /identity/v3/projects/b01d8810c1db4ccf8c89975ce0524785/users/038efbc1a3f04f378194dca84bdc1490/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:45.183310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7fe46-a94c-402e-80ac-d050d1f96daf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.186261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1250/2512] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:43:44 2022] POST /identity/v3/projects => generated 385 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:45.187307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be168e55-1070-44c1-9be8-22227bd5dcac admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.272436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be168e55-1070-44c1-9be8-22227bd5dcac admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.278859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e202f660-3569-4737-a926-c07c0af48276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.279542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202f660-3569-4737-a926-c07c0af48276 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.280012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202f660-3569-4737-a926-c07c0af48276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.280360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e202f660-3569-4737-a926-c07c0af48276 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.281224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202f660-3569-4737-a926-c07c0af48276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.284689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e202f660-3569-4737-a926-c07c0af48276 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.285909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7fe46-a94c-402e-80ac-d050d1f96daf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.291913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015a05-90ff-48da-944d-eb6042ef1fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.292803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015a05-90ff-48da-944d-eb6042ef1fcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.293252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015a05-90ff-48da-944d-eb6042ef1fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.293730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7015a05-90ff-48da-944d-eb6042ef1fcb admin admin] PATH_INFO: `/v3/projects/dc21568194bb4facbc9c971671462aa3/users/e7e9578eec534990ba636c779ca93259/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.296228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7015a05-90ff-48da-944d-eb6042ef1fcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc21568194bb4facbc9c971671462aa3, user_id=e7e9578eec534990ba636c779ca93259, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.299107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7015a05-90ff-48da-944d-eb6042ef1fcb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.309261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1251/2513] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:43:45 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:43:45.310833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c719fc-ce92-451b-a235-a70274036f69 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.311117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c719fc-ce92-451b-a235-a70274036f69 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.311393 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c719fc-ce92-451b-a235-a70274036f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.312162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1263/2514] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:45 2022] PUT /identity/v3/projects/dc21568194bb4facbc9c971671462aa3/users/e7e9578eec534990ba636c779ca93259/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:45.315789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015a05-90ff-48da-944d-eb6042ef1fcb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.358326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f8c719fc-ce92-451b-a235-a70274036f69 None None] MFA Rules not processed for user `038efbc1a3f04f378194dca84bdc1490`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:45.359881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f8c719fc-ce92-451b-a235-a70274036f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:45.385030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:45.464665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7015a05-90ff-48da-944d-eb6042ef1fcb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.469681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1252/2515] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:45.472134 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33329809-3221-4e05-83d2-c84af732f22c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.472776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33329809-3221-4e05-83d2-c84af732f22c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.473054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33329809-3221-4e05-83d2-c84af732f22c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.473319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33329809-3221-4e05-83d2-c84af732f22c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.474345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33329809-3221-4e05-83d2-c84af732f22c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.475343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c719fc-ce92-451b-a235-a70274036f69 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.476183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33329809-3221-4e05-83d2-c84af732f22c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.478247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f8c719fc-ce92-451b-a235-a70274036f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:45.507286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1264/2516] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:45.509421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33329809-3221-4e05-83d2-c84af732f22c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.528608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33329809-3221-4e05-83d2-c84af732f22c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.536201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ad497-53cd-48a0-b889-07d4786eec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.537144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ad497-53cd-48a0-b889-07d4786eec42 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.537571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ad497-53cd-48a0-b889-07d4786eec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.538029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ad497-53cd-48a0-b889-07d4786eec42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.538920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ad497-53cd-48a0-b889-07d4786eec42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.541373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ad497-53cd-48a0-b889-07d4786eec42 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.555981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1265/2517] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:43:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:43:45.560260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ad497-53cd-48a0-b889-07d4786eec42 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.572043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ad497-53cd-48a0-b889-07d4786eec42 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.578157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6f238-35f1-4d5a-80ec-c0bba9544749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.579063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6f238-35f1-4d5a-80ec-c0bba9544749 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.579495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6f238-35f1-4d5a-80ec-c0bba9544749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.579917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6f238-35f1-4d5a-80ec-c0bba9544749 admin admin] PATH_INFO: `/v3/projects/b73c89289a004f2490aa75e0d4a2b77d/users/7e989891bb9d408b9df8ca36a2e8490c/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.599762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6f238-35f1-4d5a-80ec-c0bba9544749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73c89289a004f2490aa75e0d4a2b77d, user_id=7e989891bb9d408b9df8ca36a2e8490c, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.603316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6f238-35f1-4d5a-80ec-c0bba9544749 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.604461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c719fc-ce92-451b-a235-a70274036f69 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.616247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.617597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.619169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.619842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.624228 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:45.636733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1266/2518] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:43:45 2022] PUT /identity/v3/projects/b73c89289a004f2490aa75e0d4a2b77d/users/7e989891bb9d408b9df8ca36a2e8490c/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:43:45.639549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4e519-082a-4007-b4d6-af0b0698d79d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.639936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4e519-082a-4007-b4d6-af0b0698d79d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.640236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4e519-082a-4007-b4d6-af0b0698d79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.682412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3fa4e519-082a-4007-b4d6-af0b0698d79d None None] MFA Rules not processed for user `7e989891bb9d408b9df8ca36a2e8490c`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:45.684036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3fa4e519-082a-4007-b4d6-af0b0698d79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:45.744317 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.745953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.762991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1253/2519] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:43:45.772917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:45.839878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1267/2520] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:45.863071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:45.865066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:45.879841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63340fa8-798a-45a1-a872-4d11b8b20eab service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:45.968408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793d84f-b1f6-459e-ba4a-f591827bf2fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:45.969043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d84f-b1f6-459e-ba4a-f591827bf2fb service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:45.969373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d84f-b1f6-459e-ba4a-f591827bf2fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:45.969696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793d84f-b1f6-459e-ba4a-f591827bf2fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:45.981904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793d84f-b1f6-459e-ba4a-f591827bf2fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:45.983477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d793d84f-b1f6-459e-ba4a-f591827bf2fb service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:45.996584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1254/2521] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:48.876216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4e519-082a-4007-b4d6-af0b0698d79d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:48.890069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4e519-082a-4007-b4d6-af0b0698d79d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:48.908188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc224226-55d9-418f-afee-83c53b4cf8f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:48.909254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224226-55d9-418f-afee-83c53b4cf8f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:48.914609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224226-55d9-418f-afee-83c53b4cf8f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:48.914950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc224226-55d9-418f-afee-83c53b4cf8f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:48.922908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bc224226-55d9-418f-afee-83c53b4cf8f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:49.043952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc224226-55d9-418f-afee-83c53b4cf8f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:49.045810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc224226-55d9-418f-afee-83c53b4cf8f4 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:49.062378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1268/2522] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:48 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:50.131255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793d84f-b1f6-459e-ba4a-f591827bf2fb service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:50.142461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d793d84f-b1f6-459e-ba4a-f591827bf2fb service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:50.153951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:50.154643 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:50.154918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:50.155184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:50.188829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:50.335928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:50.337689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:50.352341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1255/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:50 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:51.744649 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a14bd-f581-4a85-8b45-9f5a1904cfad None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:51.745194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a14bd-f581-4a85-8b45-9f5a1904cfad None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:51.745194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984a14bd-f581-4a85-8b45-9f5a1904cfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:51.774464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-984a14bd-f581-4a85-8b45-9f5a1904cfad None None] MFA Rules not processed for user `e7e9578eec534990ba636c779ca93259`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:43:51.775700 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-984a14bd-f581-4a85-8b45-9f5a1904cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:51.854512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1269/2524] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:43:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:43:51.869933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:51.871462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:51.958366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93269f-bc2a-4010-bc38-3b6cd3cdda77 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:51.964862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ccbdd-8b33-41a6-a7e8-c11f8f7d4114 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:51.965840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ccbdd-8b33-41a6-a7e8-c11f8f7d4114 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:51.966114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ccbdd-8b33-41a6-a7e8-c11f8f7d4114 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:51.966752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ccbdd-8b33-41a6-a7e8-c11f8f7d4114 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:51.968820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-122ccbdd-8b33-41a6-a7e8-c11f8f7d4114 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:43:52.034458 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ccbdd-8b33-41a6-a7e8-c11f8f7d4114 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:52.036111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ccbdd-8b33-41a6-a7e8-c11f8f7d4114 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:52.045755 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1256/2525] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:43:51 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:43:59.836580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a14bd-f581-4a85-8b45-9f5a1904cfad None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:43:59.908913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984a14bd-f581-4a85-8b45-9f5a1904cfad None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:43:59.916697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:43:59.917366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:43:59.917658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:43:59.917932 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] PATH_INFO: `/v3/users/32c6de95541c47cbbbb1ebc2c2e724c0` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:43:59.919068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] RBAC: Authorizing `identity:delete_user(user_id=32c6de95541c47cbbbb1ebc2c2e724c0)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:43:59.921245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:43:59.926917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:43:59.978510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32c6de95541c47cbbbb1ebc2c2e724c0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:59.987858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32c6de95541c47cbbbb1ebc2c2e724c0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:59.988618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32c6de95541c47cbbbb1ebc2c2e724c0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:43:59.997701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32c6de95541c47cbbbb1ebc2c2e724c0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.010318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32c6de95541c47cbbbb1ebc2c2e724c0'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.017377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1270/2526] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:43:59 2022] DELETE /identity/v3/users/32c6de95541c47cbbbb1ebc2c2e724c0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:00.021620 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46ed6d-f787-41f7-9230-918751500d5a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:00.022129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46ed6d-f787-41f7-9230-918751500d5a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:00.022223 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46ed6d-f787-41f7-9230-918751500d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:00.044178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0e46ed6d-f787-41f7-9230-918751500d5a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:00.045690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0e46ed6d-f787-41f7-9230-918751500d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:00.086335 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:00.108524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1257/2527] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:00.121782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:00.123252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:00.183594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4751511-944e-466f-bee2-2da854b5753c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:00.187427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:00.188038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:00.188311 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:00.188576 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:00.197736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:00.199152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:00.210972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1271/2528] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:00.259199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46ed6d-f787-41f7-9230-918751500d5a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:00.261030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0e46ed6d-f787-41f7-9230-918751500d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:00.275238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46ed6d-f787-41f7-9230-918751500d5a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:00.280353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:00.281059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:00.281336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:00.281603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] PATH_INFO: `/v3/users/7e989891bb9d408b9df8ca36a2e8490c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:00.287045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e989891bb9d408b9df8ca36a2e8490c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:00.291495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:00.377729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e989891bb9d408b9df8ca36a2e8490c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.387479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e989891bb9d408b9df8ca36a2e8490c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.388855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e989891bb9d408b9df8ca36a2e8490c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.409102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e989891bb9d408b9df8ca36a2e8490c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.433543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-69a8a93c-0f63-453d-859a-f9f1e0bc0a4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e989891bb9d408b9df8ca36a2e8490c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.434055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:00.435441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:00.441550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1258/2529] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:00 2022] DELETE /identity/v3/users/7e989891bb9d408b9df8ca36a2e8490c => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:00.446526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35698706-1f21-4ccc-a797-435196553111 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:00.447367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35698706-1f21-4ccc-a797-435196553111 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:00.447795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35698706-1f21-4ccc-a797-435196553111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:00.478604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-35698706-1f21-4ccc-a797-435196553111 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:00.480278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35698706-1f21-4ccc-a797-435196553111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:00.605500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1259/2530] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:44:00.618293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35698706-1f21-4ccc-a797-435196553111 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:00.622296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11faa7a7-2ab2-4420-bcce-6eb7d9ec751c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:00.622872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-35698706-1f21-4ccc-a797-435196553111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:00.657778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:00.716492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35698706-1f21-4ccc-a797-435196553111 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:00.721571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:00.722401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:00.722613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:00.722880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:00.730936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:00.731881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:00.732166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:00.732371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:00.734399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:00.734978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:00.736724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:00.742931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:00.751060 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1260/2531] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:44:00.752016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:00.754097 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:00.755593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:00.756171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:00.769518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1272/2532] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:44:00.771881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974d52b-7b2c-420a-b245-745fcd9c42e9 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:00.781303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:00.782075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:00.782351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:00.782619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] PATH_INFO: `/v3/projects/80a339e545ea4dc9be93617f26c91448` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:00.783843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a339e545ea4dc9be93617f26c91448)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:00.785797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:00.870751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a339e545ea4dc9be93617f26c91448', 'request_id': 'req-40424d0c-a570-4123-b491-cda4c8978b88'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.878451 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a339e545ea4dc9be93617f26c91448', 'request_id': 'req-40424d0c-a570-4123-b491-cda4c8978b88'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.886236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a339e545ea4dc9be93617f26c91448', 'request_id': 'req-40424d0c-a570-4123-b491-cda4c8978b88'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.915832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] The token cache is being invalidate because project 80a339e545ea4dc9be93617f26c91448 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:00.916306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.918392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:00.920449 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1261/2533] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:00 2022] DELETE /identity/v3/projects/80a339e545ea4dc9be93617f26c91448 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:00.924496 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:00.944811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20190e9-e102-4c2f-b73f-a5153825a120 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.027966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.028622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.029024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.029300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] PATH_INFO: `/v3/users/fd2f7daae20b465ebe9ba0b3a39db49c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.030610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd2f7daae20b465ebe9ba0b3a39db49c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.032662 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.100902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd2f7daae20b465ebe9ba0b3a39db49c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.109255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd2f7daae20b465ebe9ba0b3a39db49c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.109976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd2f7daae20b465ebe9ba0b3a39db49c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.118630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd2f7daae20b465ebe9ba0b3a39db49c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.132998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:01.134403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:01.136309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8d05712e-bdb6-4dcb-a823-89960abdcdf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd2f7daae20b465ebe9ba0b3a39db49c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.145759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1273/2534] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:00 2022] DELETE /identity/v3/users/fd2f7daae20b465ebe9ba0b3a39db49c => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:01.149229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40424d0c-a570-4123-b491-cda4c8978b88 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.247281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.247919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.248207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.249130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] PATH_INFO: `/v3/projects/b73c89289a004f2490aa75e0d4a2b77d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.250537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73c89289a004f2490aa75e0d4a2b77d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.252729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.323842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73c89289a004f2490aa75e0d4a2b77d', 'request_id': 'req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.335300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73c89289a004f2490aa75e0d4a2b77d', 'request_id': 'req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.345537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73c89289a004f2490aa75e0d4a2b77d', 'request_id': 'req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.362618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] The token cache is being invalidate because project b73c89289a004f2490aa75e0d4a2b77d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:01.362914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.363562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-41bcb7b7-c8d3-4f32-87e6-4d1b811e1dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.365082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1262/2535] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:01 2022] DELETE /identity/v3/projects/b73c89289a004f2490aa75e0d4a2b77d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:01.370480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec75803-13e2-4538-9377-e1f7a7472338 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.370771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec75803-13e2-4538-9377-e1f7a7472338 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.371047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec75803-13e2-4538-9377-e1f7a7472338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.399491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9ec75803-13e2-4538-9377-e1f7a7472338 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:01.399847 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9ec75803-13e2-4538-9377-e1f7a7472338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:01.426538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:01.472999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1274/2536] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:01.482526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f944e-19db-4b29-a256-cc745a59f613 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.482907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f944e-19db-4b29-a256-cc745a59f613 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.483243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f944e-19db-4b29-a256-cc745a59f613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.483354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec75803-13e2-4538-9377-e1f7a7472338 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:01.485154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9ec75803-13e2-4538-9377-e1f7a7472338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:01.506036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec75803-13e2-4538-9377-e1f7a7472338 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.514887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.516149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.516474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.516779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] PATH_INFO: `/v3/projects/a2392e8568a74658bb3a6a97a7fcc541` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.518521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2392e8568a74658bb3a6a97a7fcc541)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.520516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.521919 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a19f944e-19db-4b29-a256-cc745a59f613 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:01.523225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a19f944e-19db-4b29-a256-cc745a59f613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:01.531281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1263/2537] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:01.535905 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f944e-19db-4b29-a256-cc745a59f613 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:01.545415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f944e-19db-4b29-a256-cc745a59f613 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.550325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.551059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.551334 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.551602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.552248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:44:01.552550 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.554198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.555784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:01.568096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1264/2538] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:44:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 05 14:44:01.570187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2392e8568a74658bb3a6a97a7fcc541', 'request_id': 'req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.575706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:01.576185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2392e8568a74658bb3a6a97a7fcc541', 'request_id': 'req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.585682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2392e8568a74658bb3a6a97a7fcc541', 'request_id': 'req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.587182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75a43c7-c090-43c1-b5db-1777232def46 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.598355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] The token cache is being invalidate because project a2392e8568a74658bb3a6a97a7fcc541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:01.598653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.599394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.601219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1275/2539] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:01 2022] DELETE /identity/v3/projects/a2392e8568a74658bb3a6a97a7fcc541 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:01.606625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:01.608282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:01.680227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.681543 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.681849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.682118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.683134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.685353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.714025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d97e0a7-89b9-460f-8ebf-abb49e374a7f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.716859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1265/2540] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:01 2022] POST /identity/v3/projects => generated 377 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:01.721256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:01.723243 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.723386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:01.723914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.724182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.724445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] PATH_INFO: `/v3/users/038efbc1a3f04f378194dca84bdc1490` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.725706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=038efbc1a3f04f378194dca84bdc1490)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.728172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.790454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '038efbc1a3f04f378194dca84bdc1490'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.795904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:01.797330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba98f9e0-abac-498c-8c8b-55f36b31e6a4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.798521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '038efbc1a3f04f378194dca84bdc1490'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.802343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.803011 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.803343 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.803613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.804284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.806278 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.806787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '038efbc1a3f04f378194dca84bdc1490'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.815927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '038efbc1a3f04f378194dca84bdc1490'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.838691 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '038efbc1a3f04f378194dca84bdc1490'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:01.840748 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1266/2541] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:01 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:44:01.845820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:01.846739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:01.847163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1276/2542] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:01 2022] DELETE /identity/v3/users/038efbc1a3f04f378194dca84bdc1490 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:01.859554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689ae22a-bbb4-43e5-aeb1-431df6c6ff03 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.922703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f7125-1cc5-42d0-8c44-699c36f16ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.923378 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f7125-1cc5-42d0-8c44-699c36f16ae6 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.923603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f7125-1cc5-42d0-8c44-699c36f16ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.923866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f7125-1cc5-42d0-8c44-699c36f16ae6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.924631 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f7125-1cc5-42d0-8c44-699c36f16ae6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.926353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685f7125-1cc5-42d0-8c44-699c36f16ae6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.945149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1267/2543] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:44:01.952832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:01.970415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be83cdc8-f15b-4a54-b42b-1d6d07fcedc3 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:01.976599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc8d3d-fc5c-4199-a638-6931780fdcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:01.977460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8d3d-fc5c-4199-a638-6931780fdcc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:01.977812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8d3d-fc5c-4199-a638-6931780fdcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:01.978122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc8d3d-fc5c-4199-a638-6931780fdcc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:01.978936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc8d3d-fc5c-4199-a638-6931780fdcc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:01.980682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc8d3d-fc5c-4199-a638-6931780fdcc4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:01.997669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1277/2544] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:02.001441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f7125-1cc5-42d0-8c44-699c36f16ae6 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.010450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:02.019105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685f7125-1cc5-42d0-8c44-699c36f16ae6 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.038741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.039474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.039756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.040029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] PATH_INFO: `/v3/projects/af2e1681990b4f2dae32c0753bede855/users/dbd96f742bf249d29b9b298da7fc9e3b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.056611 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af2e1681990b4f2dae32c0753bede855, user_id=dbd96f742bf249d29b9b298da7fc9e3b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.059386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.076158 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1268/2545] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:02 2022] PUT /identity/v3/projects/af2e1681990b4f2dae32c0753bede855/users/dbd96f742bf249d29b9b298da7fc9e3b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:02.086918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46ac4-c5a5-4e76-b36e-6581f5e3e74c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.087257 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46ac4-c5a5-4e76-b36e-6581f5e3e74c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.087584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46ac4-c5a5-4e76-b36e-6581f5e3e74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.119677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-0ba46ac4-c5a5-4e76-b36e-6581f5e3e74c None None] MFA Rules not processed for user `dbd96f742bf249d29b9b298da7fc9e3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:02.121213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ba46ac4-c5a5-4e76-b36e-6581f5e3e74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:02.174864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.177426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:02.241990 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1278/2546] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:02.243018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46ac4-c5a5-4e76-b36e-6581f5e3e74c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.244876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0ba46ac4-c5a5-4e76-b36e-6581f5e3e74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:02.302569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164a0df-f32f-4a9d-a106-b00e157d0258 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.315417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.315417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.315417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.315417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] PATH_INFO: `/v3/projects/b01d8810c1db4ccf8c89975ce0524785` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.316362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01d8810c1db4ccf8c89975ce0524785)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.318577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.324655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46ac4-c5a5-4e76-b36e-6581f5e3e74c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.329282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223b0ac-9136-4da1-ae98-6e34f99c1fbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.329936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b0ac-9136-4da1-ae98-6e34f99c1fbf service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.330211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b0ac-9136-4da1-ae98-6e34f99c1fbf service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.330502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223b0ac-9136-4da1-ae98-6e34f99c1fbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.340348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223b0ac-9136-4da1-ae98-6e34f99c1fbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.342533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c223b0ac-9136-4da1-ae98-6e34f99c1fbf service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.353308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1279/2547] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:02.416125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01d8810c1db4ccf8c89975ce0524785', 'request_id': 'req-b62174f2-67d3-4b33-93b8-4ba21613209c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:02.422614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01d8810c1db4ccf8c89975ce0524785', 'request_id': 'req-b62174f2-67d3-4b33-93b8-4ba21613209c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:02.429686 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01d8810c1db4ccf8c89975ce0524785', 'request_id': 'req-b62174f2-67d3-4b33-93b8-4ba21613209c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:02.444753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] The token cache is being invalidate because project b01d8810c1db4ccf8c89975ce0524785 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:02.445143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:02.445949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b62174f2-67d3-4b33-93b8-4ba21613209c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:02.447540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1269/2548] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:02 2022] DELETE /identity/v3/projects/b01d8810c1db4ccf8c89975ce0524785 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:02.456871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ab44c-bf5e-43e3-977f-c3f35eff6aef None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.456871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ab44c-bf5e-43e3-977f-c3f35eff6aef None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.456871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ab44c-bf5e-43e3-977f-c3f35eff6aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.475883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-aa0ab44c-bf5e-43e3-977f-c3f35eff6aef None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:02.477088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aa0ab44c-bf5e-43e3-977f-c3f35eff6aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:02.537301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:02.554786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1280/2549] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:02.563137 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1858-6f29-4eb5-af14-1990f23d2483 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.563596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1858-6f29-4eb5-af14-1990f23d2483 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.563710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47d1858-6f29-4eb5-af14-1990f23d2483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.588636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c47d1858-6f29-4eb5-af14-1990f23d2483 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:02.590444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c47d1858-6f29-4eb5-af14-1990f23d2483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:02.598593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1270/2550] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:02.603125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ab44c-bf5e-43e3-977f-c3f35eff6aef None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.613549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ab44c-bf5e-43e3-977f-c3f35eff6aef None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.618600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.619214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.619484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.619747 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.620396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:44:02.620692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.622367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.634293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1281/2551] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:44:02.638812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d1858-6f29-4eb5-af14-1990f23d2483 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.648776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47d1858-6f29-4eb5-af14-1990f23d2483 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.655889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a760818-275e-4986-9dc2-ea8e55111da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.657129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760818-275e-4986-9dc2-ea8e55111da7 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.657785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760818-275e-4986-9dc2-ea8e55111da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.658233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a760818-275e-4986-9dc2-ea8e55111da7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.659003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a760818-275e-4986-9dc2-ea8e55111da7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.661090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a760818-275e-4986-9dc2-ea8e55111da7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.705050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1271/2552] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:02 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:44:02.709499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.723238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6705f8-652d-4bb7-ad90-634ba4988f07 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.790096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7564c-c48b-4bd4-89a9-1b7d5d941188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.790714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7564c-c48b-4bd4-89a9-1b7d5d941188 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.790989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7564c-c48b-4bd4-89a9-1b7d5d941188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.791258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c7564c-c48b-4bd4-89a9-1b7d5d941188 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.791924 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7564c-c48b-4bd4-89a9-1b7d5d941188 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.793876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c7564c-c48b-4bd4-89a9-1b7d5d941188 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.817325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1282/2553] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:02 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:02.822396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a760818-275e-4986-9dc2-ea8e55111da7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.833299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a760818-275e-4986-9dc2-ea8e55111da7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.842373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8107d19-aa35-40a8-b6db-488823f7b959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.842859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8107d19-aa35-40a8-b6db-488823f7b959 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.843170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8107d19-aa35-40a8-b6db-488823f7b959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.843439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8107d19-aa35-40a8-b6db-488823f7b959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.844077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8107d19-aa35-40a8-b6db-488823f7b959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.845675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8107d19-aa35-40a8-b6db-488823f7b959 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.856219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1272/2554] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:02.860303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7564c-c48b-4bd4-89a9-1b7d5d941188 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.870865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c7564c-c48b-4bd4-89a9-1b7d5d941188 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.875895 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045df78-9cf3-43a6-95fc-460d32d84e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.876491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045df78-9cf3-43a6-95fc-460d32d84e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.876767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045df78-9cf3-43a6-95fc-460d32d84e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.877032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045df78-9cf3-43a6-95fc-460d32d84e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.877723 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045df78-9cf3-43a6-95fc-460d32d84e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.879345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045df78-9cf3-43a6-95fc-460d32d84e54 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.890010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1283/2555] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:02.894341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8107d19-aa35-40a8-b6db-488823f7b959 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:02.904988 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8107d19-aa35-40a8-b6db-488823f7b959 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:02.910081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:02.910726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.911005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.911275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] PATH_INFO: `/v3/projects/bea771b3ff124fc8b9b9ac57f8ea12b4/users/005d0f9318534cfc98c37d822e49bdc4/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.927117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=bea771b3ff124fc8b9b9ac57f8ea12b4, user_id=005d0f9318534cfc98c37d822e49bdc4, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:02.929802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:02.944183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1273/2556] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:02 2022] PUT /identity/v3/projects/bea771b3ff124fc8b9b9ac57f8ea12b4/users/005d0f9318534cfc98c37d822e49bdc4/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:02.955725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab770ac-36d4-4709-b659-cb2c875d194d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:02.956209 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab770ac-36d4-4709-b659-cb2c875d194d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:02.956339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab770ac-36d4-4709-b659-cb2c875d194d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:02.985675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-fab770ac-36d4-4709-b659-cb2c875d194d None None] MFA Rules not processed for user `005d0f9318534cfc98c37d822e49bdc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:02.986898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fab770ac-36d4-4709-b659-cb2c875d194d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:03.055801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1284/2557] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:03.066913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:03.068768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:03.080074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:03.141409 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e0b11-080a-4f92-b71d-79d1ad6936ff admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:03.145190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:03.145865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:03.146103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:03.146374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:03.155931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:03.157433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:03.168107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1274/2558] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:04.594977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329e9aa-26d0-4552-914c-2feec8eff97f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:04.595575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329e9aa-26d0-4552-914c-2feec8eff97f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:04.595833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329e9aa-26d0-4552-914c-2feec8eff97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:04.615458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7329e9aa-26d0-4552-914c-2feec8eff97f None None] MFA Rules not processed for user `005d0f9318534cfc98c37d822e49bdc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:04.616682 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7329e9aa-26d0-4552-914c-2feec8eff97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:04.627683 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1285/2559] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:12.804143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:12.806372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:12.872179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e70bc-e74d-4930-b7ad-e2bf884221f3 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:12.877209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:12.878368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:12.878702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:12.879026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:12.881763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:12.951155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:12.952595 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:12.962319 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1275/2560] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:14.610222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:16.689499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329e9aa-26d0-4552-914c-2feec8eff97f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:16.693392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7329e9aa-26d0-4552-914c-2feec8eff97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:16.709236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329e9aa-26d0-4552-914c-2feec8eff97f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:16.716323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:16.716969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:16.717269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:16.717737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] PATH_INFO: `/v3/users/6e4a13a6981f4e0898dc22ea8525a1ee` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:16.719152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e4a13a6981f4e0898dc22ea8525a1ee)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:16.721078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:16.770224 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e4a13a6981f4e0898dc22ea8525a1ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:16.777325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e4a13a6981f4e0898dc22ea8525a1ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:16.778271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e4a13a6981f4e0898dc22ea8525a1ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:16.787667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e4a13a6981f4e0898dc22ea8525a1ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:16.801207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-65a69861-56e1-4aa9-89aa-4b7e3c8f7259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e4a13a6981f4e0898dc22ea8525a1ee'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:16.807862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1286/2561] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:16 2022] DELETE /identity/v3/users/6e4a13a6981f4e0898dc22ea8525a1ee => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:16.991049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:17.005141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc5de6-e5e6-44f6-be5b-9360cd9337ff service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:17.068186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:17.068833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.069138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.069404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] PATH_INFO: `/v3/projects/ae3d0e9a5de8481fa19a58f7780a4b7a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.070577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3d0e9a5de8481fa19a58f7780a4b7a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:17.072513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:17.075054 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:17.271651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'request_id': 'req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:17.278666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'request_id': 'req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:17.286408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3d0e9a5de8481fa19a58f7780a4b7a', 'request_id': 'req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:17.302277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] The token cache is being invalidate because project ae3d0e9a5de8481fa19a58f7780a4b7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:17.302745 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:17.303501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1a3a3e74-2b8b-4c87-87c3-bbdfe187b03f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:17.305265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1276/2562] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:16 2022] DELETE /identity/v3/projects/ae3d0e9a5de8481fa19a58f7780a4b7a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:17.313169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab4b8a-f089-46f2-b281-aa5a5ee6acd1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.313451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab4b8a-f089-46f2-b281-aa5a5ee6acd1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.313746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab4b8a-f089-46f2-b281-aa5a5ee6acd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.336040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e8ab4b8a-f089-46f2-b281-aa5a5ee6acd1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:17.337313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e8ab4b8a-f089-46f2-b281-aa5a5ee6acd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:17.421761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1287/2563] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:17.425901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab06192-8d67-4c7a-b53a-b32c5e391042 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.426184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab06192-8d67-4c7a-b53a-b32c5e391042 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.426575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab06192-8d67-4c7a-b53a-b32c5e391042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.451951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-8ab06192-8d67-4c7a-b53a-b32c5e391042 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:17.455465 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8ab06192-8d67-4c7a-b53a-b32c5e391042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:17.470911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1277/2564] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:17.474942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab4b8a-f089-46f2-b281-aa5a5ee6acd1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:17.486731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab4b8a-f089-46f2-b281-aa5a5ee6acd1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:17.493984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:17.495006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.495595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.497189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.498356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:44:17.498812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:17.500744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:17.513448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1288/2565] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:44:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:44:17.517947 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab06192-8d67-4c7a-b53a-b32c5e391042 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:17.542043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab06192-8d67-4c7a-b53a-b32c5e391042 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:17.548500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00cfa0-9439-4cde-9554-77ac5150b51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:17.549482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00cfa0-9439-4cde-9554-77ac5150b51a admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.549864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00cfa0-9439-4cde-9554-77ac5150b51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.550133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00cfa0-9439-4cde-9554-77ac5150b51a admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.550888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00cfa0-9439-4cde-9554-77ac5150b51a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:17.554382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f00cfa0-9439-4cde-9554-77ac5150b51a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:17.594143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1278/2566] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:17 2022] POST /identity/v3/projects => generated 403 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:17.598390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:17.610398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a850b5c-0f02-47b7-9a18-1cfc27a60be0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:17.675702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853851a-9e01-4bff-8a69-5aa987155e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:17.676328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853851a-9e01-4bff-8a69-5aa987155e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.676610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853851a-9e01-4bff-8a69-5aa987155e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.676879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0853851a-9e01-4bff-8a69-5aa987155e8d admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.677515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853851a-9e01-4bff-8a69-5aa987155e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:17.679506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0853851a-9e01-4bff-8a69-5aa987155e8d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:17.691712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:17.709454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1289/2567] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:17 2022] POST /identity/v3/users => generated 302 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:44:17.713522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00cfa0-9439-4cde-9554-77ac5150b51a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:17.724242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f00cfa0-9439-4cde-9554-77ac5150b51a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:17.729031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebfdfe-4a1d-40c0-9981-a21bb4b9e32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:17.729936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebfdfe-4a1d-40c0-9981-a21bb4b9e32e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.730219 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebfdfe-4a1d-40c0-9981-a21bb4b9e32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.730491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ebfdfe-4a1d-40c0-9981-a21bb4b9e32e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.731143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebfdfe-4a1d-40c0-9981-a21bb4b9e32e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:17.732740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ebfdfe-4a1d-40c0-9981-a21bb4b9e32e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:17.745936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1279/2568] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:17.747090 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853851a-9e01-4bff-8a69-5aa987155e8d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:17.757774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0853851a-9e01-4bff-8a69-5aa987155e8d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:17.762659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1c013e-f81b-4aba-90d6-4a541216b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:17.763274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c013e-f81b-4aba-90d6-4a541216b9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.763546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c013e-f81b-4aba-90d6-4a541216b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.763812 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1c013e-f81b-4aba-90d6-4a541216b9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.764455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c013e-f81b-4aba-90d6-4a541216b9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:17.766107 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1c013e-f81b-4aba-90d6-4a541216b9b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:17.776531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1290/2569] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:17.780775 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebfdfe-4a1d-40c0-9981-a21bb4b9e32e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:17.791089 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ebfdfe-4a1d-40c0-9981-a21bb4b9e32e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:17.796584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:17.797285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.797560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.797860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] PATH_INFO: `/v3/projects/4d891c5285934373bd8d91642af0d65a/users/b5ca90ad8854438baa4c90cab02f46f2/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.812977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d891c5285934373bd8d91642af0d65a, user_id=b5ca90ad8854438baa4c90cab02f46f2, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:17.815635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:17.830107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1280/2570] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:17 2022] PUT /identity/v3/projects/4d891c5285934373bd8d91642af0d65a/users/b5ca90ad8854438baa4c90cab02f46f2/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:17.835193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0148f-1b3c-4e9a-9982-c40109b9e1e1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.835533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0148f-1b3c-4e9a-9982-c40109b9e1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.835868 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d0148f-1b3c-4e9a-9982-c40109b9e1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:17.855883 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-49d0148f-1b3c-4e9a-9982-c40109b9e1e1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:17.857099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-49d0148f-1b3c-4e9a-9982-c40109b9e1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:17.920763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1291/2571] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:17.930758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:17.933218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:17.994146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678cf727-a03f-43c5-9c5f-5d8385b1a4c1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:17.998277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435792d-eb2b-4e6b-af14-f839651f5278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:17.998912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435792d-eb2b-4e6b-af14-f839651f5278 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:17.999105 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435792d-eb2b-4e6b-af14-f839651f5278 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:17.999373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4435792d-eb2b-4e6b-af14-f839651f5278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:18.009448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435792d-eb2b-4e6b-af14-f839651f5278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:18.011053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4435792d-eb2b-4e6b-af14-f839651f5278 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:18.021053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1281/2572] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:18.655733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4688c94-b4cf-4c92-80e3-17da13750434 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:18.656201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4688c94-b4cf-4c92-80e3-17da13750434 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:18.656633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4688c94-b4cf-4c92-80e3-17da13750434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:18.697579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f4688c94-b4cf-4c92-80e3-17da13750434 None None] MFA Rules not processed for user `b5ca90ad8854438baa4c90cab02f46f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:18.699473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f4688c94-b4cf-4c92-80e3-17da13750434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:18.777235 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1292/2573] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:18 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:18.781874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435792d-eb2b-4e6b-af14-f839651f5278 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:18.792862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4435792d-eb2b-4e6b-af14-f839651f5278 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:18.797585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9a7ef-a9dd-4865-b32d-2f00c2e47e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:18.798226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9a7ef-a9dd-4865-b32d-2f00c2e47e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:18.798503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9a7ef-a9dd-4865-b32d-2f00c2e47e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:18.798774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9a7ef-a9dd-4865-b32d-2f00c2e47e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:18.799568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9a7ef-a9dd-4865-b32d-2f00c2e47e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:18.801578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9a7ef-a9dd-4865-b32d-2f00c2e47e4b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:18.830591 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1282/2574] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:18 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:18.837157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4688c94-b4cf-4c92-80e3-17da13750434 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:18.848537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4688c94-b4cf-4c92-80e3-17da13750434 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:18.911015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9cf62-1b68-4697-8ed1-90be1a9df0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:18.911708 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9cf62-1b68-4697-8ed1-90be1a9df0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:18.911983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9cf62-1b68-4697-8ed1-90be1a9df0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:18.912254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9cf62-1b68-4697-8ed1-90be1a9df0d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:18.912882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9cf62-1b68-4697-8ed1-90be1a9df0d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:18.914887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9cf62-1b68-4697-8ed1-90be1a9df0d5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:18.932386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:18.941695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1293/2575] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:18 2022] POST /identity/v3/users => generated 309 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:18.942980 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9a7ef-a9dd-4865-b32d-2f00c2e47e4b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:18.953388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9a7ef-a9dd-4865-b32d-2f00c2e47e4b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:18.958579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f07970-2a0a-4465-9836-d272fc28c7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:18.959171 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f07970-2a0a-4465-9836-d272fc28c7e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:18.959444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f07970-2a0a-4465-9836-d272fc28c7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:18.959708 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f07970-2a0a-4465-9836-d272fc28c7e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:18.960344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f07970-2a0a-4465-9836-d272fc28c7e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:18.961943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f07970-2a0a-4465-9836-d272fc28c7e4 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:18.963348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:18.974052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1283/2576] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:18.978352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9cf62-1b68-4697-8ed1-90be1a9df0d5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:18.989484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9cf62-1b68-4697-8ed1-90be1a9df0d5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:18.994223 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ebfb7-d366-4e17-aa31-89aab7d759a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:18.994875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ebfb7-d366-4e17-aa31-89aab7d759a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:18.995292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ebfb7-d366-4e17-aa31-89aab7d759a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:18.995560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7ebfb7-d366-4e17-aa31-89aab7d759a2 admin admin] PATH_INFO: `/v3/projects/3bca0c85831b4c51b758c5cbf7eea2e3/users/8085a8b77f4a478f98de2633e6f17ed9/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:19.014738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ebfb7-d366-4e17-aa31-89aab7d759a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bca0c85831b4c51b758c5cbf7eea2e3, user_id=8085a8b77f4a478f98de2633e6f17ed9, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:19.017506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7ebfb7-d366-4e17-aa31-89aab7d759a2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:19.032499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1294/2577] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:18 2022] PUT /identity/v3/projects/3bca0c85831b4c51b758c5cbf7eea2e3/users/8085a8b77f4a478f98de2633e6f17ed9/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:19.036568 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f07970-2a0a-4465-9836-d272fc28c7e4 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:19.046783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f07970-2a0a-4465-9836-d272fc28c7e4 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:19.105359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8119ed-23b6-4ac0-b3bf-5f9d4427a84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:19.106016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8119ed-23b6-4ac0-b3bf-5f9d4427a84c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:19.106292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8119ed-23b6-4ac0-b3bf-5f9d4427a84c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:19.106557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8119ed-23b6-4ac0-b3bf-5f9d4427a84c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:19.107203 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8119ed-23b6-4ac0-b3bf-5f9d4427a84c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:19.108785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8119ed-23b6-4ac0-b3bf-5f9d4427a84c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:19.118604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1284/2578] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:44:19.122901 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ebfb7-d366-4e17-aa31-89aab7d759a2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:19.133309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7ebfb7-d366-4e17-aa31-89aab7d759a2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:19.138472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152660b-67a4-4642-8b2d-db1235d57e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:19.139125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152660b-67a4-4642-8b2d-db1235d57e36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:19.139402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152660b-67a4-4642-8b2d-db1235d57e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:19.139674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3152660b-67a4-4642-8b2d-db1235d57e36 admin admin] PATH_INFO: `/v3/projects/3bca0c85831b4c51b758c5cbf7eea2e3/users/8085a8b77f4a478f98de2633e6f17ed9/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:19.141607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152660b-67a4-4642-8b2d-db1235d57e36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bca0c85831b4c51b758c5cbf7eea2e3, user_id=8085a8b77f4a478f98de2633e6f17ed9, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:19.144335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3152660b-67a4-4642-8b2d-db1235d57e36 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:19.154819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1295/2579] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:19 2022] PUT /identity/v3/projects/3bca0c85831b4c51b758c5cbf7eea2e3/users/8085a8b77f4a478f98de2633e6f17ed9/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:19.808281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff72eb-2661-4413-a879-4376f3d4b63b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:19.808827 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff72eb-2661-4413-a879-4376f3d4b63b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:19.809193 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff72eb-2661-4413-a879-4376f3d4b63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:19.843562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-fdff72eb-2661-4413-a879-4376f3d4b63b None None] MFA Rules not processed for user `8085a8b77f4a478f98de2633e6f17ed9`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:19.845223 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fdff72eb-2661-4413-a879-4376f3d4b63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:19.928517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1285/2580] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:19 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:20.980506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152660b-67a4-4642-8b2d-db1235d57e36 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:21.043930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3152660b-67a4-4642-8b2d-db1235d57e36 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:21.048511 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:21.049147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:21.049421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:21.049709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] PATH_INFO: `/v3/users/b5ca90ad8854438baa4c90cab02f46f2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:21.050816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ca90ad8854438baa4c90cab02f46f2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:21.052768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:21.111566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ca90ad8854438baa4c90cab02f46f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.119042 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ca90ad8854438baa4c90cab02f46f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.119821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ca90ad8854438baa4c90cab02f46f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.136502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ca90ad8854438baa4c90cab02f46f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.148905 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ca90ad8854438baa4c90cab02f46f2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.155159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1296/2581] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:20 2022] DELETE /identity/v3/users/b5ca90ad8854438baa4c90cab02f46f2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:21.326169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff72eb-2661-4413-a879-4376f3d4b63b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:21.336529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff72eb-2661-4413-a879-4376f3d4b63b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:21.396648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:21.397192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:21.397462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:21.397765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] PATH_INFO: `/v3/projects/4d891c5285934373bd8d91642af0d65a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:21.399081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d891c5285934373bd8d91642af0d65a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:21.401066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:21.423432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:21.449741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d891c5285934373bd8d91642af0d65a', 'request_id': 'req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.455062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d891c5285934373bd8d91642af0d65a', 'request_id': 'req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.461424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d891c5285934373bd8d91642af0d65a', 'request_id': 'req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.474815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] The token cache is being invalidate because project 4d891c5285934373bd8d91642af0d65a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:21.475095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.475693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.477149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1286/2582] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:21 2022] DELETE /identity/v3/projects/4d891c5285934373bd8d91642af0d65a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:21.481220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:21.482675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:21.496229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5270e8-d49c-4e0c-abbe-b93b49a4d621 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:21.542465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:21.563469 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:21.564106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:21.564374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:21.564637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] PATH_INFO: `/v3/users/8085a8b77f4a478f98de2633e6f17ed9` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:21.565807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8085a8b77f4a478f98de2633e6f17ed9)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:21.568104 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:21.623616 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8085a8b77f4a478f98de2633e6f17ed9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.631404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8085a8b77f4a478f98de2633e6f17ed9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.632133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8085a8b77f4a478f98de2633e6f17ed9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.641946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8085a8b77f4a478f98de2633e6f17ed9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.654339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-6293bf6d-08d8-437e-8886-44bb5546f0a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8085a8b77f4a478f98de2633e6f17ed9'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.660825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1297/2583] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:21 2022] DELETE /identity/v3/users/8085a8b77f4a478f98de2633e6f17ed9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:21.838102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:21.840040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:21.856478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea66df-3ae7-413d-9a7e-5dc21e603eb7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:21.921424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:21.922010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:21.922284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:21.922556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] PATH_INFO: `/v3/projects/3bca0c85831b4c51b758c5cbf7eea2e3` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:21.923655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bca0c85831b4c51b758c5cbf7eea2e3)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:21.925557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:21.976516 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bca0c85831b4c51b758c5cbf7eea2e3', 'request_id': 'req-f104169c-4f9c-4b59-b9a6-e20720628d94'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.982240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bca0c85831b4c51b758c5cbf7eea2e3', 'request_id': 'req-f104169c-4f9c-4b59-b9a6-e20720628d94'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:21.988453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bca0c85831b4c51b758c5cbf7eea2e3', 'request_id': 'req-f104169c-4f9c-4b59-b9a6-e20720628d94'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:22.003362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] The token cache is being invalidate because project 3bca0c85831b4c51b758c5cbf7eea2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:22.003682 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:22.004329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-f104169c-4f9c-4b59-b9a6-e20720628d94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:22.005866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1287/2584] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:21 2022] DELETE /identity/v3/projects/3bca0c85831b4c51b758c5cbf7eea2e3 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:22.011110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3d55-2d48-4c5b-a46d-606e1a6f2c57 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.011591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3d55-2d48-4c5b-a46d-606e1a6f2c57 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.011725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f3d55-2d48-4c5b-a46d-606e1a6f2c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.032609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5c1f3d55-2d48-4c5b-a46d-606e1a6f2c57 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:22.033849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5c1f3d55-2d48-4c5b-a46d-606e1a6f2c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:22.093194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1298/2585] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:22.100928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8239fc2-437a-4a2c-8afd-b06b7806adbd None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.101209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8239fc2-437a-4a2c-8afd-b06b7806adbd None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.101482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8239fc2-437a-4a2c-8afd-b06b7806adbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.121671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-b8239fc2-437a-4a2c-8afd-b06b7806adbd None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:22.122881 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b8239fc2-437a-4a2c-8afd-b06b7806adbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:22.130962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1288/2586] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:22.135752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f3d55-2d48-4c5b-a46d-606e1a6f2c57 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.146378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f3d55-2d48-4c5b-a46d-606e1a6f2c57 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.151551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.152165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.152438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.152739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.153710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:44:22.154005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.155589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.168147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1299/2587] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:44:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:44:22.173249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8239fc2-437a-4a2c-8afd-b06b7806adbd None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.182705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8239fc2-437a-4a2c-8afd-b06b7806adbd None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.188344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8304f-58ed-403a-88e8-3554423bcf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.188944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8304f-58ed-403a-88e8-3554423bcf3b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.189213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8304f-58ed-403a-88e8-3554423bcf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.189474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c8304f-58ed-403a-88e8-3554423bcf3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.190152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8304f-58ed-403a-88e8-3554423bcf3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.192063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c8304f-58ed-403a-88e8-3554423bcf3b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.224953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1289/2588] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:22 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:44:22.229614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.240109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca71d5c-7f61-453b-bab4-6f22b28530c5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.287771 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:22.302922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65df62-913c-4274-9b22-a973f48da4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.303537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65df62-913c-4274-9b22-a973f48da4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.303819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65df62-913c-4274-9b22-a973f48da4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.304089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec65df62-913c-4274-9b22-a973f48da4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.304755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65df62-913c-4274-9b22-a973f48da4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.306760 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec65df62-913c-4274-9b22-a973f48da4e8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.330536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1300/2589] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:22 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:44:22.334837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8304f-58ed-403a-88e8-3554423bcf3b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.346183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c8304f-58ed-403a-88e8-3554423bcf3b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.354236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e1fc5-6a52-4186-9c14-f913d5b3f365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.354978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e1fc5-6a52-4186-9c14-f913d5b3f365 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.355254 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e1fc5-6a52-4186-9c14-f913d5b3f365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.355572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e1fc5-6a52-4186-9c14-f913d5b3f365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.356231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e1fc5-6a52-4186-9c14-f913d5b3f365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.357839 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e1fc5-6a52-4186-9c14-f913d5b3f365 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.360176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:22.373888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1290/2590] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:22.374929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65df62-913c-4274-9b22-a973f48da4e8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.385707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec65df62-913c-4274-9b22-a973f48da4e8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.390229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443d8980-54fa-4692-9f3c-3c85c4f01b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.390823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8980-54fa-4692-9f3c-3c85c4f01b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.391095 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8980-54fa-4692-9f3c-3c85c4f01b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.391358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443d8980-54fa-4692-9f3c-3c85c4f01b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.392038 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d8980-54fa-4692-9f3c-3c85c4f01b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.393619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443d8980-54fa-4692-9f3c-3c85c4f01b1c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.404133 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1301/2591] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:44:22.409471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e1fc5-6a52-4186-9c14-f913d5b3f365 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.419797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e1fc5-6a52-4186-9c14-f913d5b3f365 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.424483 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a4599-2d13-4a4b-96e2-8d7aad1d891b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.425076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a4599-2d13-4a4b-96e2-8d7aad1d891b admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.425354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a4599-2d13-4a4b-96e2-8d7aad1d891b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.425624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367a4599-2d13-4a4b-96e2-8d7aad1d891b admin admin] PATH_INFO: `/v3/projects/21cd6b09a4d0429c8f34851fca265000/users/e2880a88ab4a427597b8ce4d70140e1b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.442403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a4599-2d13-4a4b-96e2-8d7aad1d891b admin admin] RBAC: Authorizing `identity:create_grant(project_id=21cd6b09a4d0429c8f34851fca265000, user_id=e2880a88ab4a427597b8ce4d70140e1b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.445076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367a4599-2d13-4a4b-96e2-8d7aad1d891b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.461237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1291/2592] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:22 2022] PUT /identity/v3/projects/21cd6b09a4d0429c8f34851fca265000/users/e2880a88ab4a427597b8ce4d70140e1b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:22.471627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ea28-c276-4ce8-960a-7a75d158735c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.471998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ea28-c276-4ce8-960a-7a75d158735c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.472372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6ea28-c276-4ce8-960a-7a75d158735c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.505228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-baa6ea28-c276-4ce8-960a-7a75d158735c None None] MFA Rules not processed for user `e2880a88ab4a427597b8ce4d70140e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:22.506628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-baa6ea28-c276-4ce8-960a-7a75d158735c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:22.573897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1302/2593] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:22.578428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a4599-2d13-4a4b-96e2-8d7aad1d891b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.648706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367a4599-2d13-4a4b-96e2-8d7aad1d891b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.655153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a9678-146b-498a-82c9-14f331137343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.655759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a9678-146b-498a-82c9-14f331137343 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.656030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a9678-146b-498a-82c9-14f331137343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.656295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0a9678-146b-498a-82c9-14f331137343 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.656925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a9678-146b-498a-82c9-14f331137343 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.658854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0a9678-146b-498a-82c9-14f331137343 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.688858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1292/2594] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:22 2022] POST /identity/v3/projects => generated 369 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:22.695479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6ea28-c276-4ce8-960a-7a75d158735c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.717149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6ea28-c276-4ce8-960a-7a75d158735c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.794805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe879cc-7c0e-4e26-9be2-07f5bd772ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.795433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe879cc-7c0e-4e26-9be2-07f5bd772ace admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.795703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe879cc-7c0e-4e26-9be2-07f5bd772ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.795970 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe879cc-7c0e-4e26-9be2-07f5bd772ace admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.796598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe879cc-7c0e-4e26-9be2-07f5bd772ace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.798641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe879cc-7c0e-4e26-9be2-07f5bd772ace admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.821699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1303/2595] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:22 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:22.825344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a9678-146b-498a-82c9-14f331137343 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.837145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0a9678-146b-498a-82c9-14f331137343 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.843401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b97f54-3930-49a9-8bdd-ba544744a0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.844003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b97f54-3930-49a9-8bdd-ba544744a0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.844275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b97f54-3930-49a9-8bdd-ba544744a0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.844542 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b97f54-3930-49a9-8bdd-ba544744a0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.845173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b97f54-3930-49a9-8bdd-ba544744a0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.846815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b97f54-3930-49a9-8bdd-ba544744a0f2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.858043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1293/2596] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:22.862481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe879cc-7c0e-4e26-9be2-07f5bd772ace admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.873048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe879cc-7c0e-4e26-9be2-07f5bd772ace admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.879287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71355c-b9c6-4510-b6ee-0b4ef498c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.879991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71355c-b9c6-4510-b6ee-0b4ef498c59a admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.880269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71355c-b9c6-4510-b6ee-0b4ef498c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.880520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca71355c-b9c6-4510-b6ee-0b4ef498c59a admin admin] PATH_INFO: `/v3/projects/2ba07d21777a49b3a94cc5ebf759480e/users/5a112a0123db48a388b7dd902a3ec65b/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.897083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71355c-b9c6-4510-b6ee-0b4ef498c59a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba07d21777a49b3a94cc5ebf759480e, user_id=5a112a0123db48a388b7dd902a3ec65b, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.899837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca71355c-b9c6-4510-b6ee-0b4ef498c59a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.914882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1304/2597] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:22 2022] PUT /identity/v3/projects/2ba07d21777a49b3a94cc5ebf759480e/users/5a112a0123db48a388b7dd902a3ec65b/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:22.918766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b97f54-3930-49a9-8bdd-ba544744a0f2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:22.928582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b97f54-3930-49a9-8bdd-ba544744a0f2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:22.984665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4df0c6-3028-46e0-88e6-292a9cf43b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:22.985192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df0c6-3028-46e0-88e6-292a9cf43b05 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:22.985466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df0c6-3028-46e0-88e6-292a9cf43b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:22.985768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4df0c6-3028-46e0-88e6-292a9cf43b05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:22.986405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4df0c6-3028-46e0-88e6-292a9cf43b05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:22.988022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4df0c6-3028-46e0-88e6-292a9cf43b05 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:22.998205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1294/2598] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:22 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:44:23.002629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71355c-b9c6-4510-b6ee-0b4ef498c59a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:23.013472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca71355c-b9c6-4510-b6ee-0b4ef498c59a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:23.018377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:23.019088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:23.019362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:23.019635 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] PATH_INFO: `/v3/projects/2ba07d21777a49b3a94cc5ebf759480e/users/5a112a0123db48a388b7dd902a3ec65b/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:23.021858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba07d21777a49b3a94cc5ebf759480e, user_id=5a112a0123db48a388b7dd902a3ec65b, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:23.024545 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:23.036336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1305/2599] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:23 2022] PUT /identity/v3/projects/2ba07d21777a49b3a94cc5ebf759480e/users/5a112a0123db48a388b7dd902a3ec65b/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:23.048006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c9050-1d07-44a7-a852-8d31d4458e07 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:23.048299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c9050-1d07-44a7-a852-8d31d4458e07 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:23.048566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c9050-1d07-44a7-a852-8d31d4458e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:23.078725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-db4c9050-1d07-44a7-a852-8d31d4458e07 None None] MFA Rules not processed for user `5a112a0123db48a388b7dd902a3ec65b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:23.079942 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-db4c9050-1d07-44a7-a852-8d31d4458e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:23.120798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:23.163432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1295/2600] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:23 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:23.178343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:23.180139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:23.192281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:23.242825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f56890b-1644-4830-beda-b2bd2cb61af5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:23.246452 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bc0351-ba9d-49ca-bffb-d027e3963ccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:23.247054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc0351-ba9d-49ca-bffb-d027e3963ccf service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:23.247323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc0351-ba9d-49ca-bffb-d027e3963ccf service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:23.247589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bc0351-ba9d-49ca-bffb-d027e3963ccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:23.256346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc0351-ba9d-49ca-bffb-d027e3963ccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:23.257921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bc0351-ba9d-49ca-bffb-d027e3963ccf service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:23.269035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1306/2601] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:23 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:24.755131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c9050-1d07-44a7-a852-8d31d4458e07 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:24.827269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c9050-1d07-44a7-a852-8d31d4458e07 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:24.832577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:24.833204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:24.833472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:24.833742 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] PATH_INFO: `/v3/users/e2880a88ab4a427597b8ce4d70140e1b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:24.835044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2880a88ab4a427597b8ce4d70140e1b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:24.836986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:24.890946 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2880a88ab4a427597b8ce4d70140e1b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:24.899345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2880a88ab4a427597b8ce4d70140e1b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:24.900095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2880a88ab4a427597b8ce4d70140e1b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:24.914448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2880a88ab4a427597b8ce4d70140e1b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:24.927111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2880a88ab4a427597b8ce4d70140e1b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:24.933047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1296/2602] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:24 2022] DELETE /identity/v3/users/e2880a88ab4a427597b8ce4d70140e1b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:24.937591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6121d3c-7f7b-4587-92cf-731236ba652c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:24.938183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6121d3c-7f7b-4587-92cf-731236ba652c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:24.938538 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6121d3c-7f7b-4587-92cf-731236ba652c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:24.958512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e6121d3c-7f7b-4587-92cf-731236ba652c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:24.959978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e6121d3c-7f7b-4587-92cf-731236ba652c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:25.049096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1307/2603] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:25.059014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:25.060448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:25.116590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:25.122845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967d6b7-e63b-4b22-a61e-dc07f4543a87 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:25.130500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7ec5c-2810-4e67-a0c9-c79e7c8a4978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:25.131430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7ec5c-2810-4e67-a0c9-c79e7c8a4978 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:25.131851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7ec5c-2810-4e67-a0c9-c79e7c8a4978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:25.132240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc7ec5c-2810-4e67-a0c9-c79e7c8a4978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:25.144078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7ec5c-2810-4e67-a0c9-c79e7c8a4978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:25.146921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc7ec5c-2810-4e67-a0c9-c79e7c8a4978 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:25.161571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1297/2604] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:44:25.450509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6121d3c-7f7b-4587-92cf-731236ba652c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:25.452215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e6121d3c-7f7b-4587-92cf-731236ba652c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:25.466118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6121d3c-7f7b-4587-92cf-731236ba652c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:25.471459 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:25.472202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:25.472580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:25.472913 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] PATH_INFO: `/v3/projects/21cd6b09a4d0429c8f34851fca265000` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:25.474287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] RBAC: Authorizing `identity:delete_project(project_id=21cd6b09a4d0429c8f34851fca265000)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:25.476647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:25.527733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21cd6b09a4d0429c8f34851fca265000', 'request_id': 'req-d5415536-29bf-45d4-9a71-51d727b8d2df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.533179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21cd6b09a4d0429c8f34851fca265000', 'request_id': 'req-d5415536-29bf-45d4-9a71-51d727b8d2df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.540093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21cd6b09a4d0429c8f34851fca265000', 'request_id': 'req-d5415536-29bf-45d4-9a71-51d727b8d2df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.554699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] The token cache is being invalidate because project 21cd6b09a4d0429c8f34851fca265000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:25.554989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.555615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.557238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1308/2605] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:25 2022] DELETE /identity/v3/projects/21cd6b09a4d0429c8f34851fca265000 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:25.561185 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7ec5c-2810-4e67-a0c9-c79e7c8a4978 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:25.570993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc7ec5c-2810-4e67-a0c9-c79e7c8a4978 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:25.629455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:25.630176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:25.630443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:25.630702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] PATH_INFO: `/v3/users/5a112a0123db48a388b7dd902a3ec65b` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:25.631987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a112a0123db48a388b7dd902a3ec65b)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:25.633922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:25.690461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a112a0123db48a388b7dd902a3ec65b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.697100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a112a0123db48a388b7dd902a3ec65b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.697790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a112a0123db48a388b7dd902a3ec65b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.705917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a112a0123db48a388b7dd902a3ec65b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.717878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-df58bc87-e3b1-4803-802f-fbdca960f23b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a112a0123db48a388b7dd902a3ec65b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:25.724392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1298/2606] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:25 2022] DELETE /identity/v3/users/5a112a0123db48a388b7dd902a3ec65b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:26.067591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.069212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:26.089334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5415536-29bf-45d4-9a71-51d727b8d2df admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:26.122063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:26.156810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:26.156810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.156810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.156810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] PATH_INFO: `/v3/projects/2ba07d21777a49b3a94cc5ebf759480e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.157178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba07d21777a49b3a94cc5ebf759480e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:26.157178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:26.205659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba07d21777a49b3a94cc5ebf759480e', 'request_id': 'req-8fd19a08-0250-4034-906f-1ab79801a278'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:26.212093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba07d21777a49b3a94cc5ebf759480e', 'request_id': 'req-8fd19a08-0250-4034-906f-1ab79801a278'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:26.222324 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba07d21777a49b3a94cc5ebf759480e', 'request_id': 'req-8fd19a08-0250-4034-906f-1ab79801a278'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:26.236731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] The token cache is being invalidate because project 2ba07d21777a49b3a94cc5ebf759480e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:26.237083 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:26.237799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8fd19a08-0250-4034-906f-1ab79801a278 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:26.239738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1309/2607] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:26 2022] DELETE /identity/v3/projects/2ba07d21777a49b3a94cc5ebf759480e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:26.244671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6aca-9ba7-4497-a770-8283e3b0afa0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.245268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6aca-9ba7-4497-a770-8283e3b0afa0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.245366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc6aca-9ba7-4497-a770-8283e3b0afa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.265038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d3bc6aca-9ba7-4497-a770-8283e3b0afa0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:26.266455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d3bc6aca-9ba7-4497-a770-8283e3b0afa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:26.325784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1299/2608] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:26.333705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a9fba-e05f-41df-9488-288d281fe79b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.333988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a9fba-e05f-41df-9488-288d281fe79b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.334252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094a9fba-e05f-41df-9488-288d281fe79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.354128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-094a9fba-e05f-41df-9488-288d281fe79b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:26.355327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-094a9fba-e05f-41df-9488-288d281fe79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:26.364716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1310/2609] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:44:26.368697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc6aca-9ba7-4497-a770-8283e3b0afa0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.378658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc6aca-9ba7-4497-a770-8283e3b0afa0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:26.383547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:26.384180 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.384459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.384719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.385402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:44:26.385770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:26.387326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:26.394347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:26.400590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1300/2610] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:44:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:44:26.408780 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a9fba-e05f-41df-9488-288d281fe79b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.424214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094a9fba-e05f-41df-9488-288d281fe79b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:26.431202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30251a4-0a63-4319-8a90-e17c3d645c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:26.432076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30251a4-0a63-4319-8a90-e17c3d645c59 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.432537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30251a4-0a63-4319-8a90-e17c3d645c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.432918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30251a4-0a63-4319-8a90-e17c3d645c59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.433915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30251a4-0a63-4319-8a90-e17c3d645c59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:26.436939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30251a4-0a63-4319-8a90-e17c3d645c59 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:26.472241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1311/2611] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:26 2022] POST /identity/v3/projects => generated 391 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:26.478823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.492179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b6659-09f3-4376-87cf-52a43cd3cb08 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:26.555243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cd26f-cd5e-4f9e-a1b6-9e0fe198ac5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:26.555870 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cd26f-cd5e-4f9e-a1b6-9e0fe198ac5b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.556157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cd26f-cd5e-4f9e-a1b6-9e0fe198ac5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.556427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cd26f-cd5e-4f9e-a1b6-9e0fe198ac5b admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.557084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cd26f-cd5e-4f9e-a1b6-9e0fe198ac5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:26.559123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41cd26f-cd5e-4f9e-a1b6-9e0fe198ac5b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:26.582621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1301/2612] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:26 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:26.586965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30251a4-0a63-4319-8a90-e17c3d645c59 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.598325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30251a4-0a63-4319-8a90-e17c3d645c59 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:26.604297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75457-8038-4ff6-bfca-bc2173ca5d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:26.604920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75457-8038-4ff6-bfca-bc2173ca5d61 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.605198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75457-8038-4ff6-bfca-bc2173ca5d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.605583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b75457-8038-4ff6-bfca-bc2173ca5d61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.606360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b75457-8038-4ff6-bfca-bc2173ca5d61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:26.608127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b75457-8038-4ff6-bfca-bc2173ca5d61 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:26.618505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1312/2613] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:26.622609 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cd26f-cd5e-4f9e-a1b6-9e0fe198ac5b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.632665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cd26f-cd5e-4f9e-a1b6-9e0fe198ac5b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:26.637923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a780ec-1aee-4543-a109-9f526db0a687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:26.638519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a780ec-1aee-4543-a109-9f526db0a687 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.638790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a780ec-1aee-4543-a109-9f526db0a687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.639080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a780ec-1aee-4543-a109-9f526db0a687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.639720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a780ec-1aee-4543-a109-9f526db0a687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:26.641298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a780ec-1aee-4543-a109-9f526db0a687 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:26.652341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1302/2614] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:26.656535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75457-8038-4ff6-bfca-bc2173ca5d61 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.667093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b75457-8038-4ff6-bfca-bc2173ca5d61 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:26.672803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35eb12-a34f-4fff-ac9c-e0ea37e78d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:26.673456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35eb12-a34f-4fff-ac9c-e0ea37e78d1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.673768 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35eb12-a34f-4fff-ac9c-e0ea37e78d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.674037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea35eb12-a34f-4fff-ac9c-e0ea37e78d1f admin admin] PATH_INFO: `/v3/projects/b7615814fa394acd8496a6724a7ef802/users/437cb5ac26f24f9d91d7b5db4a67faca/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.689764 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35eb12-a34f-4fff-ac9c-e0ea37e78d1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7615814fa394acd8496a6724a7ef802, user_id=437cb5ac26f24f9d91d7b5db4a67faca, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:26.692488 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea35eb12-a34f-4fff-ac9c-e0ea37e78d1f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:26.706317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1313/2615] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:26 2022] PUT /identity/v3/projects/b7615814fa394acd8496a6724a7ef802/users/437cb5ac26f24f9d91d7b5db4a67faca/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:26.718524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddf84-a6b3-4ad6-9148-3b4f1ff9f46c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.718801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddf84-a6b3-4ad6-9148-3b4f1ff9f46c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.719036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ddf84-a6b3-4ad6-9148-3b4f1ff9f46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.746710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4e1ddf84-a6b3-4ad6-9148-3b4f1ff9f46c None None] MFA Rules not processed for user `437cb5ac26f24f9d91d7b5db4a67faca`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:26.747949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4e1ddf84-a6b3-4ad6-9148-3b4f1ff9f46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:26.838366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1303/2616] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:26.846199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35eb12-a34f-4fff-ac9c-e0ea37e78d1f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.912762 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea35eb12-a34f-4fff-ac9c-e0ea37e78d1f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:26.918182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e7288-af99-4afe-b8c2-f6f613e42faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:26.918893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7288-af99-4afe-b8c2-f6f613e42faf admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:26.919260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7288-af99-4afe-b8c2-f6f613e42faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:26.919615 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e7288-af99-4afe-b8c2-f6f613e42faf admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:26.920479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e7288-af99-4afe-b8c2-f6f613e42faf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:26.922560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e7288-af99-4afe-b8c2-f6f613e42faf admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:26.927460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:26.954844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1314/2617] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:26 2022] POST /identity/v3/projects => generated 393 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:26.958614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ddf84-a6b3-4ad6-9148-3b4f1ff9f46c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:26.970314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ddf84-a6b3-4ad6-9148-3b4f1ff9f46c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:27.030799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec9976-a63d-47f9-8312-69cb450898c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:27.031441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec9976-a63d-47f9-8312-69cb450898c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:27.031717 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec9976-a63d-47f9-8312-69cb450898c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:27.032012 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec9976-a63d-47f9-8312-69cb450898c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:27.032657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec9976-a63d-47f9-8312-69cb450898c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:27.034635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec9976-a63d-47f9-8312-69cb450898c1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:27.046128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:27.058267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1304/2618] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:26 2022] POST /identity/v3/users => generated 303 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:44:27.062255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e7288-af99-4afe-b8c2-f6f613e42faf admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:27.072857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e7288-af99-4afe-b8c2-f6f613e42faf admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:27.077767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626788f-1bbf-4af5-865a-251ef8282e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:27.078322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626788f-1bbf-4af5-865a-251ef8282e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:27.078597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626788f-1bbf-4af5-865a-251ef8282e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:27.078862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3626788f-1bbf-4af5-865a-251ef8282e29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:27.079507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626788f-1bbf-4af5-865a-251ef8282e29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:27.081138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3626788f-1bbf-4af5-865a-251ef8282e29 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:27.091856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1315/2619] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:27.096407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec9976-a63d-47f9-8312-69cb450898c1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:27.107371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec9976-a63d-47f9-8312-69cb450898c1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:27.113937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8e81e-9867-4250-8e0e-4ee29678f0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:27.114831 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8e81e-9867-4250-8e0e-4ee29678f0a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:27.115116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8e81e-9867-4250-8e0e-4ee29678f0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:27.115434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8e81e-9867-4250-8e0e-4ee29678f0a1 admin admin] PATH_INFO: `/v3/projects/53f6137cbfe34d3f953ecf8f91e24fce/users/d976a456be9c44d08fda3faf8cab9caf/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:27.132342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8e81e-9867-4250-8e0e-4ee29678f0a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f6137cbfe34d3f953ecf8f91e24fce, user_id=d976a456be9c44d08fda3faf8cab9caf, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:27.135038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8e81e-9867-4250-8e0e-4ee29678f0a1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:27.154740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1305/2620] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:27 2022] PUT /identity/v3/projects/53f6137cbfe34d3f953ecf8f91e24fce/users/d976a456be9c44d08fda3faf8cab9caf/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:27.156111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626788f-1bbf-4af5-865a-251ef8282e29 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:27.171882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3626788f-1bbf-4af5-865a-251ef8282e29 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:27.240823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005643f2-3913-4a50-9d07-d0b5966d5bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:27.241437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005643f2-3913-4a50-9d07-d0b5966d5bca admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:27.241744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005643f2-3913-4a50-9d07-d0b5966d5bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:27.242292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005643f2-3913-4a50-9d07-d0b5966d5bca admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:27.242952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005643f2-3913-4a50-9d07-d0b5966d5bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:27.244456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005643f2-3913-4a50-9d07-d0b5966d5bca admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:27.254600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1316/2621] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:27.258545 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8e81e-9867-4250-8e0e-4ee29678f0a1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:27.269279 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8e81e-9867-4250-8e0e-4ee29678f0a1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:27.274290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:27.274963 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:27.275240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:27.275508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] PATH_INFO: `/v3/projects/53f6137cbfe34d3f953ecf8f91e24fce/users/d976a456be9c44d08fda3faf8cab9caf/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:27.278282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53f6137cbfe34d3f953ecf8f91e24fce, user_id=d976a456be9c44d08fda3faf8cab9caf, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:27.280205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:27.291267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1306/2622] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:27 2022] PUT /identity/v3/projects/53f6137cbfe34d3f953ecf8f91e24fce/users/d976a456be9c44d08fda3faf8cab9caf/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:27.301466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dba28-b159-448a-aa30-40fbff109055 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:27.301781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dba28-b159-448a-aa30-40fbff109055 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:27.302050 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060dba28-b159-448a-aa30-40fbff109055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:27.329422 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-060dba28-b159-448a-aa30-40fbff109055 None None] MFA Rules not processed for user `d976a456be9c44d08fda3faf8cab9caf`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:27.330645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-060dba28-b159-448a-aa30-40fbff109055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:27.400854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1317/2623] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:27 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:27.413684 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:27.415018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:27.477000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe8695-80db-42b7-9b51-9f98083d6993 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:27.480887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b991b21-91f7-4e9f-b388-ada6ce08743d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:27.481498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b991b21-91f7-4e9f-b388-ada6ce08743d service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:27.481807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b991b21-91f7-4e9f-b388-ada6ce08743d service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:27.482076 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b991b21-91f7-4e9f-b388-ada6ce08743d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:27.492283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b991b21-91f7-4e9f-b388-ada6ce08743d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:27.508592 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b991b21-91f7-4e9f-b388-ada6ce08743d service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:27.518672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1307/2624] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:27 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:27.601222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dba28-b159-448a-aa30-40fbff109055 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:27.601673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060dba28-b159-448a-aa30-40fbff109055 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:27.605477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1deb48-1341-47bb-b462-a90f582df65d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:27.606202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1deb48-1341-47bb-b462-a90f582df65d service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:27.606482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1deb48-1341-47bb-b462-a90f582df65d service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:27.606752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1deb48-1341-47bb-b462-a90f582df65d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:27.660873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:27.666443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1deb48-1341-47bb-b462-a90f582df65d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:27.668623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1deb48-1341-47bb-b462-a90f582df65d service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:27.678202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1318/2625] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:37.411405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b991b21-91f7-4e9f-b388-ada6ce08743d service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:37.503063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b991b21-91f7-4e9f-b388-ada6ce08743d service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:37.512651 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:37.514916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:37.515233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:37.515541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] PATH_INFO: `/v3/users/437cb5ac26f24f9d91d7b5db4a67faca` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:37.516889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=437cb5ac26f24f9d91d7b5db4a67faca)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:37.518820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:37.570042 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '437cb5ac26f24f9d91d7b5db4a67faca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:37.582819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '437cb5ac26f24f9d91d7b5db4a67faca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:37.583825 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '437cb5ac26f24f9d91d7b5db4a67faca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:37.594106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '437cb5ac26f24f9d91d7b5db4a67faca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:37.595811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:37.614894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '437cb5ac26f24f9d91d7b5db4a67faca'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:37.623673 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1308/2626] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:37 2022] DELETE /identity/v3/users/437cb5ac26f24f9d91d7b5db4a67faca => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:37.628124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc5a7f-886d-4389-b285-bfd20de06eb7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:37.628525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc5a7f-886d-4389-b285-bfd20de06eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:37.639394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc5a7f-886d-4389-b285-bfd20de06eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:37.661955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4bbc5a7f-886d-4389-b285-bfd20de06eb7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:37.663307 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4bbc5a7f-886d-4389-b285-bfd20de06eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:37.739745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1319/2627] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:37.751022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:37.753328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:37.815175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49b97e9-c3ec-46f5-91ca-fd592e09ff4e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:37.818886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b3f16-cf3d-4802-92e2-37ff49b0d54f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:37.819482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3f16-cf3d-4802-92e2-37ff49b0d54f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:37.819754 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3f16-cf3d-4802-92e2-37ff49b0d54f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:37.820014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8b3f16-cf3d-4802-92e2-37ff49b0d54f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:37.829008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b3f16-cf3d-4802-92e2-37ff49b0d54f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:37.830441 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8b3f16-cf3d-4802-92e2-37ff49b0d54f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:37.840244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1309/2628] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:38.118193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc5a7f-886d-4389-b285-bfd20de06eb7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:38.119619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4bbc5a7f-886d-4389-b285-bfd20de06eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:38.132716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc5a7f-886d-4389-b285-bfd20de06eb7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:38.138259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:38.138910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:38.139220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:38.139464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] PATH_INFO: `/v3/projects/b7615814fa394acd8496a6724a7ef802` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:38.140608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7615814fa394acd8496a6724a7ef802)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:38.142627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:38.209979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7615814fa394acd8496a6724a7ef802', 'request_id': 'req-fb19c530-eebf-448c-82fe-974fc3ec55a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.215763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7615814fa394acd8496a6724a7ef802', 'request_id': 'req-fb19c530-eebf-448c-82fe-974fc3ec55a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.222016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7615814fa394acd8496a6724a7ef802', 'request_id': 'req-fb19c530-eebf-448c-82fe-974fc3ec55a4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.236489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] The token cache is being invalidate because project b7615814fa394acd8496a6724a7ef802 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:38.236869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.237609 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.239253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1320/2629] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:38 2022] DELETE /identity/v3/projects/b7615814fa394acd8496a6724a7ef802 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:38.244506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b3f16-cf3d-4802-92e2-37ff49b0d54f service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:38.259342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8b3f16-cf3d-4802-92e2-37ff49b0d54f service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:38.330663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:38.331453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:38.332601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:38.332601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] PATH_INFO: `/v3/users/d976a456be9c44d08fda3faf8cab9caf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:38.334132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d976a456be9c44d08fda3faf8cab9caf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:38.336045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:38.387246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd976a456be9c44d08fda3faf8cab9caf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.396001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd976a456be9c44d08fda3faf8cab9caf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.396840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd976a456be9c44d08fda3faf8cab9caf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.405573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd976a456be9c44d08fda3faf8cab9caf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.418662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a7c63ea4-847a-402c-969e-f8a8097392e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd976a456be9c44d08fda3faf8cab9caf'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.424639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1310/2630] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:38 2022] DELETE /identity/v3/users/d976a456be9c44d08fda3faf8cab9caf => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:38.710542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:38.712514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:38.730629 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19c530-eebf-448c-82fe-974fc3ec55a4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:38.798965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:38.799702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:38.800071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:38.800413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] PATH_INFO: `/v3/projects/53f6137cbfe34d3f953ecf8f91e24fce` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:38.801619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53f6137cbfe34d3f953ecf8f91e24fce)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:38.803737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:38.828484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:38.859369 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53f6137cbfe34d3f953ecf8f91e24fce', 'request_id': 'req-affca614-2bcb-479a-a4b7-6128a713c5d8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.865833 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53f6137cbfe34d3f953ecf8f91e24fce', 'request_id': 'req-affca614-2bcb-479a-a4b7-6128a713c5d8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.876701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53f6137cbfe34d3f953ecf8f91e24fce', 'request_id': 'req-affca614-2bcb-479a-a4b7-6128a713c5d8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.893465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] The token cache is being invalidate because project 53f6137cbfe34d3f953ecf8f91e24fce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:38.893804 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.894474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-affca614-2bcb-479a-a4b7-6128a713c5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:38.897775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1321/2631] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:38 2022] DELETE /identity/v3/projects/53f6137cbfe34d3f953ecf8f91e24fce => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:38.901798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf9f6a-7829-45c3-b238-0d4f43313483 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:38.917679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf9f6a-7829-45c3-b238-0d4f43313483 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:38.917679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcf9f6a-7829-45c3-b238-0d4f43313483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:38.917679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:38.927917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7fcf9f6a-7829-45c3-b238-0d4f43313483 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:38.930141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7fcf9f6a-7829-45c3-b238-0d4f43313483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:39.004437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1311/2632] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:44:39.018353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c89130-0718-4e28-9d14-689601abe568 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.018733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c89130-0718-4e28-9d14-689601abe568 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.019088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c89130-0718-4e28-9d14-689601abe568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.039705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-69c89130-0718-4e28-9d14-689601abe568 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:39.041012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-69c89130-0718-4e28-9d14-689601abe568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:39.049205 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1322/2633] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:39.054022 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf9f6a-7829-45c3-b238-0d4f43313483 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.065720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcf9f6a-7829-45c3-b238-0d4f43313483 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.070815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.071576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.071943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.072313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.073207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:44:39.073578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.075440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.090307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1312/2634] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:44:39.094381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c89130-0718-4e28-9d14-689601abe568 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.103652 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c89130-0718-4e28-9d14-689601abe568 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.108447 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9c187-c683-4c56-9e52-924909ca9fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.109106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9c187-c683-4c56-9e52-924909ca9fdb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.109386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9c187-c683-4c56-9e52-924909ca9fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.109674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9c187-c683-4c56-9e52-924909ca9fdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.110354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9c187-c683-4c56-9e52-924909ca9fdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.112270 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9c187-c683-4c56-9e52-924909ca9fdb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.144653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1323/2635] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:39 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:39.150756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.161064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640e4f5-0102-4a56-9ea9-105fe8899f2d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.226813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd70057-3828-4144-a4b9-404ca2948f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.227551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70057-3828-4144-a4b9-404ca2948f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.227925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70057-3828-4144-a4b9-404ca2948f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.228216 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd70057-3828-4144-a4b9-404ca2948f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.228847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd70057-3828-4144-a4b9-404ca2948f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.230811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd70057-3828-4144-a4b9-404ca2948f25 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.259189 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1313/2636] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:39 2022] POST /identity/v3/users => generated 290 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:39.260480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9c187-c683-4c56-9e52-924909ca9fdb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.271016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9c187-c683-4c56-9e52-924909ca9fdb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.275742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86955327-2143-4f36-b772-083ef51c41ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.276338 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86955327-2143-4f36-b772-083ef51c41ee admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.276610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86955327-2143-4f36-b772-083ef51c41ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.276873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86955327-2143-4f36-b772-083ef51c41ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.277508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86955327-2143-4f36-b772-083ef51c41ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.279172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86955327-2143-4f36-b772-083ef51c41ee admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.289204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1324/2637] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:44:39.293234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd70057-3828-4144-a4b9-404ca2948f25 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.303298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd70057-3828-4144-a4b9-404ca2948f25 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.308178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12728585-30a7-4774-911f-f204880b4692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.308773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728585-30a7-4774-911f-f204880b4692 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.309046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728585-30a7-4774-911f-f204880b4692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.309330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12728585-30a7-4774-911f-f204880b4692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.309991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12728585-30a7-4774-911f-f204880b4692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.311610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12728585-30a7-4774-911f-f204880b4692 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.322091 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1314/2638] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:39.326125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86955327-2143-4f36-b772-083ef51c41ee admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.336513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86955327-2143-4f36-b772-083ef51c41ee admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.344021 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3940b7-ffc0-49b7-a7d0-abb67fcd2402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.344963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3940b7-ffc0-49b7-a7d0-abb67fcd2402 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.345312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3940b7-ffc0-49b7-a7d0-abb67fcd2402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.345734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3940b7-ffc0-49b7-a7d0-abb67fcd2402 admin admin] PATH_INFO: `/v3/projects/df17ca94be91492e9ddc2951ff342435/users/990562c04e134e5e829905d0b7bb1e0b/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.365173 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3940b7-ffc0-49b7-a7d0-abb67fcd2402 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df17ca94be91492e9ddc2951ff342435, user_id=990562c04e134e5e829905d0b7bb1e0b, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.367938 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3940b7-ffc0-49b7-a7d0-abb67fcd2402 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.387985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1325/2639] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:39 2022] PUT /identity/v3/projects/df17ca94be91492e9ddc2951ff342435/users/990562c04e134e5e829905d0b7bb1e0b/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:39.400667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9471a-2cbc-4ca5-b473-136de7ed75e7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.401029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9471a-2cbc-4ca5-b473-136de7ed75e7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.401243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea9471a-2cbc-4ca5-b473-136de7ed75e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.437453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cea9471a-2cbc-4ca5-b473-136de7ed75e7 None None] MFA Rules not processed for user `990562c04e134e5e829905d0b7bb1e0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:39.438693 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cea9471a-2cbc-4ca5-b473-136de7ed75e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:39.505422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:39.511182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1315/2640] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:44:39.517156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3940b7-ffc0-49b7-a7d0-abb67fcd2402 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.588216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3940b7-ffc0-49b7-a7d0-abb67fcd2402 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.594599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35104852-5377-4ed2-8959-5e8ffe1fc4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.595413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35104852-5377-4ed2-8959-5e8ffe1fc4d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.595734 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35104852-5377-4ed2-8959-5e8ffe1fc4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.595953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35104852-5377-4ed2-8959-5e8ffe1fc4d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.596790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35104852-5377-4ed2-8959-5e8ffe1fc4d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.599026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35104852-5377-4ed2-8959-5e8ffe1fc4d0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.634406 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1326/2641] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:39 2022] POST /identity/v3/projects => generated 381 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:39.638371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9471a-2cbc-4ca5-b473-136de7ed75e7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.650108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea9471a-2cbc-4ca5-b473-136de7ed75e7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.710206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed66464-60c5-40c9-8c4f-c8822ee75e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.710772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed66464-60c5-40c9-8c4f-c8822ee75e10 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.711049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed66464-60c5-40c9-8c4f-c8822ee75e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.711318 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed66464-60c5-40c9-8c4f-c8822ee75e10 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.711962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed66464-60c5-40c9-8c4f-c8822ee75e10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.713909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed66464-60c5-40c9-8c4f-c8822ee75e10 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.741899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1316/2642] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:39 2022] POST /identity/v3/users => generated 297 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:39.743119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35104852-5377-4ed2-8959-5e8ffe1fc4d0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.755365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35104852-5377-4ed2-8959-5e8ffe1fc4d0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.760504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a208b58-71d3-4331-bd9d-ddb2158643e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.761112 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a208b58-71d3-4331-bd9d-ddb2158643e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.761391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a208b58-71d3-4331-bd9d-ddb2158643e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.761686 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a208b58-71d3-4331-bd9d-ddb2158643e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.762343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a208b58-71d3-4331-bd9d-ddb2158643e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.764046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a208b58-71d3-4331-bd9d-ddb2158643e8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.774729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1327/2643] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:39.779188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed66464-60c5-40c9-8c4f-c8822ee75e10 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.790338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed66464-60c5-40c9-8c4f-c8822ee75e10 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.796276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac65ade-5e13-4792-80bc-0277eabbc656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.796958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac65ade-5e13-4792-80bc-0277eabbc656 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.797262 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac65ade-5e13-4792-80bc-0277eabbc656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.797518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac65ade-5e13-4792-80bc-0277eabbc656 admin admin] PATH_INFO: `/v3/projects/f0216bd708dd4e168b81cff006713993/users/49251c77574f47639297d6a7c65d70c4/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.812783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac65ade-5e13-4792-80bc-0277eabbc656 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0216bd708dd4e168b81cff006713993, user_id=49251c77574f47639297d6a7c65d70c4, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.815443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac65ade-5e13-4792-80bc-0277eabbc656 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.828979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1317/2644] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:39 2022] PUT /identity/v3/projects/f0216bd708dd4e168b81cff006713993/users/49251c77574f47639297d6a7c65d70c4/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:39.833098 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a208b58-71d3-4331-bd9d-ddb2158643e8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.843724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a208b58-71d3-4331-bd9d-ddb2158643e8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.857345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:39.902687 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaa109-4c1c-49bd-905e-51b45177a8d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.903316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa109-4c1c-49bd-905e-51b45177a8d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.903592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa109-4c1c-49bd-905e-51b45177a8d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.903861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaa109-4c1c-49bd-905e-51b45177a8d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.904770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaa109-4c1c-49bd-905e-51b45177a8d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.906419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaa109-4c1c-49bd-905e-51b45177a8d0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.916170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1328/2645] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:39 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:39.920768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac65ade-5e13-4792-80bc-0277eabbc656 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:39.935563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac65ade-5e13-4792-80bc-0277eabbc656 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:39.942674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:39.943331 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.943602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.943867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] PATH_INFO: `/v3/projects/f0216bd708dd4e168b81cff006713993/users/49251c77574f47639297d6a7c65d70c4/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:39.946265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0216bd708dd4e168b81cff006713993, user_id=49251c77574f47639297d6a7c65d70c4, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:39.948895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:39.961301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1318/2646] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:39 2022] PUT /identity/v3/projects/f0216bd708dd4e168b81cff006713993/users/49251c77574f47639297d6a7c65d70c4/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:39.974618 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00c4b-4e5f-4d8f-a916-21b44f2f01e2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:39.975054 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00c4b-4e5f-4d8f-a916-21b44f2f01e2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:39.975516 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea00c4b-4e5f-4d8f-a916-21b44f2f01e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:40.009322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5ea00c4b-4e5f-4d8f-a916-21b44f2f01e2 None None] MFA Rules not processed for user `49251c77574f47639297d6a7c65d70c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:40.010718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5ea00c4b-4e5f-4d8f-a916-21b44f2f01e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:40.078363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1329/2647] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:39 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:40.089996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:40.092191 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:40.149910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151809c3-179a-4d1d-9e97-68adb47ee038 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:40.154235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4084ce-4b36-4a6e-80ad-53292225aa99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:40.154951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4084ce-4b36-4a6e-80ad-53292225aa99 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:40.155330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4084ce-4b36-4a6e-80ad-53292225aa99 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:40.155680 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4084ce-4b36-4a6e-80ad-53292225aa99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:40.165746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4084ce-4b36-4a6e-80ad-53292225aa99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:40.167281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4084ce-4b36-4a6e-80ad-53292225aa99 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:40.177702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1319/2648] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:40.562858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea00c4b-4e5f-4d8f-a916-21b44f2f01e2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:40.639546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea00c4b-4e5f-4d8f-a916-21b44f2f01e2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:40.645531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:40.647770 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:40.648119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:40.648403 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] PATH_INFO: `/v3/users/990562c04e134e5e829905d0b7bb1e0b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:40.649675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=990562c04e134e5e829905d0b7bb1e0b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:40.651647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:40.700014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db561f-4283-474b-9735-e7ab41b9711c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:40.700571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db561f-4283-474b-9735-e7ab41b9711c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:40.700678 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db561f-4283-474b-9735-e7ab41b9711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:40.709893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:40.716116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '990562c04e134e5e829905d0b7bb1e0b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:40.720863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-39db561f-4283-474b-9735-e7ab41b9711c None None] MFA Rules not processed for user `dea49d304629415895813b6ab7f9c9db`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:40.722134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39db561f-4283-474b-9735-e7ab41b9711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:40.723718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '990562c04e134e5e829905d0b7bb1e0b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:40.724358 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '990562c04e134e5e829905d0b7bb1e0b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:40.732736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '990562c04e134e5e829905d0b7bb1e0b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:40.744946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b7ed17f8-2237-49fc-9b75-a8e0c2247b42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '990562c04e134e5e829905d0b7bb1e0b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:40.751260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1330/2649] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:40 2022] DELETE /identity/v3/users/990562c04e134e5e829905d0b7bb1e0b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:40.755590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0a44c-5064-4bb8-bcfb-401c6b6fe4a4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:40.755896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0a44c-5064-4bb8-bcfb-401c6b6fe4a4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:40.756162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e0a44c-5064-4bb8-bcfb-401c6b6fe4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:40.769856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:40.777524 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-c5e0a44c-5064-4bb8-bcfb-401c6b6fe4a4 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:40.778880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c5e0a44c-5064-4bb8-bcfb-401c6b6fe4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:40.781950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1320/2650] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:40.842468 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1331/2651] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:44:40.852642 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db561f-4283-474b-9735-e7ab41b9711c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:40.854174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-39db561f-4283-474b-9735-e7ab41b9711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:40.911294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db561f-4283-474b-9735-e7ab41b9711c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:40.962951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2c9f2-8b1a-42eb-b86e-e524c11b04b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:40.963567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2c9f2-8b1a-42eb-b86e-e524c11b04b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:40.963845 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2c9f2-8b1a-42eb-b86e-e524c11b04b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:40.964114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d2c9f2-8b1a-42eb-b86e-e524c11b04b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:40.973894 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2c9f2-8b1a-42eb-b86e-e524c11b04b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:40.975333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d2c9f2-8b1a-42eb-b86e-e524c11b04b7 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:40.986501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1321/2652] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:41.364387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0a44c-5064-4bb8-bcfb-401c6b6fe4a4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:41.366077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c5e0a44c-5064-4bb8-bcfb-401c6b6fe4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:41.378899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e0a44c-5064-4bb8-bcfb-401c6b6fe4a4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:41.384852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:41.385520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:41.385820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:41.386232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] PATH_INFO: `/v3/projects/df17ca94be91492e9ddc2951ff342435` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:41.387392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=df17ca94be91492e9ddc2951ff342435)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:41.389870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:41.446398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df17ca94be91492e9ddc2951ff342435', 'request_id': 'req-4bbc9c2b-e2ef-494b-862d-297d35b620bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.451818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df17ca94be91492e9ddc2951ff342435', 'request_id': 'req-4bbc9c2b-e2ef-494b-862d-297d35b620bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.461819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df17ca94be91492e9ddc2951ff342435', 'request_id': 'req-4bbc9c2b-e2ef-494b-862d-297d35b620bb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.480466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] The token cache is being invalidate because project df17ca94be91492e9ddc2951ff342435 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:41.480877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.481837 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.483742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1332/2653] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:41 2022] DELETE /identity/v3/projects/df17ca94be91492e9ddc2951ff342435 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:41.488486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2c9f2-8b1a-42eb-b86e-e524c11b04b7 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:41.501571 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d2c9f2-8b1a-42eb-b86e-e524c11b04b7 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:41.569411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:41.570153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:41.570429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:41.570711 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] PATH_INFO: `/v3/users/49251c77574f47639297d6a7c65d70c4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:41.571850 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49251c77574f47639297d6a7c65d70c4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:41.574005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:41.641390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49251c77574f47639297d6a7c65d70c4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.649304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49251c77574f47639297d6a7c65d70c4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.650097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49251c77574f47639297d6a7c65d70c4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.655250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:41.664787 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49251c77574f47639297d6a7c65d70c4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.680636 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-42b8dd75-23c7-43a7-8c75-05bcb3a1e180 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49251c77574f47639297d6a7c65d70c4'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:41.687818 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1322/2654] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:41 2022] DELETE /identity/v3/users/49251c77574f47639297d6a7c65d70c4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:42.138952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.141049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:42.156115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbc9c2b-e2ef-494b-862d-297d35b620bb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.197141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:42.214481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.215125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.215400 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.215669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] PATH_INFO: `/v3/projects/f0216bd708dd4e168b81cff006713993` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.216828 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0216bd708dd4e168b81cff006713993)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.218763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.271143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0216bd708dd4e168b81cff006713993', 'request_id': 'req-284342f3-089f-4507-88bf-929b975a7b4e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.277803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0216bd708dd4e168b81cff006713993', 'request_id': 'req-284342f3-089f-4507-88bf-929b975a7b4e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.285179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0216bd708dd4e168b81cff006713993', 'request_id': 'req-284342f3-089f-4507-88bf-929b975a7b4e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.303179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] The token cache is being invalidate because project f0216bd708dd4e168b81cff006713993 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:42.303596 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.305330 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.306015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1333/2655] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:42 2022] DELETE /identity/v3/projects/f0216bd708dd4e168b81cff006713993 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:42.312538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d645cb2-5088-4a11-bd44-1b31216a310d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.313070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d645cb2-5088-4a11-bd44-1b31216a310d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.313300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d645cb2-5088-4a11-bd44-1b31216a310d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.342789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-0d645cb2-5088-4a11-bd44-1b31216a310d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:42.344256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0d645cb2-5088-4a11-bd44-1b31216a310d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:42.397746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.399543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:42.448039 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1323/2656] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:42.456061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c512b8-4d83-47ea-bd59-1f1de7d58506 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.456367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c512b8-4d83-47ea-bd59-1f1de7d58506 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.456641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c512b8-4d83-47ea-bd59-1f1de7d58506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.474592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284342f3-089f-4507-88bf-929b975a7b4e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.476694 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-27c512b8-4d83-47ea-bd59-1f1de7d58506 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:42.477952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27c512b8-4d83-47ea-bd59-1f1de7d58506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:42.479287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.479941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.480216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.480487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] PATH_INFO: `/v3/users/dbd96f742bf249d29b9b298da7fc9e3b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.481628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbd96f742bf249d29b9b298da7fc9e3b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.483679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.485989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1324/2657] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:44:42.490630 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c512b8-4d83-47ea-bd59-1f1de7d58506 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.503170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c512b8-4d83-47ea-bd59-1f1de7d58506 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.508306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.509330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.509805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.510070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.510846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:44:42.511142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.512874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.530644 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1325/2658] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:44:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:44:42.534271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.545066 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20641601-8de2-438b-a972-c93a44dd9d2e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.547795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbd96f742bf249d29b9b298da7fc9e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.550807 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.551472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.551812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.552159 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.553584 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.555251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbd96f742bf249d29b9b298da7fc9e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.555899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.556132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbd96f742bf249d29b9b298da7fc9e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.565549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbd96f742bf249d29b9b298da7fc9e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.575473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:42.581936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2fe38a89-7939-4a37-903f-963d02a7e6a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbd96f742bf249d29b9b298da7fc9e3b'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:42.591113 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1334/2660] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:44:42 2022] DELETE /identity/v3/users/dbd96f742bf249d29b9b298da7fc9e3b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:42.591313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1326/2660] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:42 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:44:42.596401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.596854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e9cc-5fa6-457a-b60f-dc98c6cfc31d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.597126 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e9cc-5fa6-457a-b60f-dc98c6cfc31d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.597399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2e9cc-5fa6-457a-b60f-dc98c6cfc31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.599087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:42.620911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ab2e9cc-5fa6-457a-b60f-dc98c6cfc31d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:42.622180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ab2e9cc-5fa6-457a-b60f-dc98c6cfc31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:42.672912 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de429a81-557a-4f58-93e8-ecfa9fc00321 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.679967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e59713-7447-42fb-adb8-5fadc1337a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.680604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e59713-7447-42fb-adb8-5fadc1337a81 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.680879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e59713-7447-42fb-adb8-5fadc1337a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.681153 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e59713-7447-42fb-adb8-5fadc1337a81 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.681844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e59713-7447-42fb-adb8-5fadc1337a81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.683802 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e59713-7447-42fb-adb8-5fadc1337a81 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.695694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1335/2661] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:42.707082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2e9cc-5fa6-457a-b60f-dc98c6cfc31d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.707817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1327/2662] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:42 2022] POST /identity/v3/users => generated 300 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:42.708486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ab2e9cc-5fa6-457a-b60f-dc98c6cfc31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:42.711774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e59713-7447-42fb-adb8-5fadc1337a81 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.721736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e59713-7447-42fb-adb8-5fadc1337a81 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.726353 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890350b8-5d6f-4f59-86f8-987890e6ba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.727120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890350b8-5d6f-4f59-86f8-987890e6ba87 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.727492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890350b8-5d6f-4f59-86f8-987890e6ba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.727982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890350b8-5d6f-4f59-86f8-987890e6ba87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.728650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890350b8-5d6f-4f59-86f8-987890e6ba87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.730431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890350b8-5d6f-4f59-86f8-987890e6ba87 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.741658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1328/2663] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:44:42.744212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890350b8-5d6f-4f59-86f8-987890e6ba87 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.753547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890350b8-5d6f-4f59-86f8-987890e6ba87 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.758752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ff2fc-1451-45da-942d-c4a99a507df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.759342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff2fc-1451-45da-942d-c4a99a507df9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.759619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff2fc-1451-45da-942d-c4a99a507df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.759888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ff2fc-1451-45da-942d-c4a99a507df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.760522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ff2fc-1451-45da-942d-c4a99a507df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.762183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ff2fc-1451-45da-942d-c4a99a507df9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.769242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2e9cc-5fa6-457a-b60f-dc98c6cfc31d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.772837 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1329/2664] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:44:42.775420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c3a4c-8683-4301-a73e-85c022631658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.776111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3a4c-8683-4301-a73e-85c022631658 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.777035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ff2fc-1451-45da-942d-c4a99a507df9 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.777604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3a4c-8683-4301-a73e-85c022631658 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.777914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c3a4c-8683-4301-a73e-85c022631658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.782606 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:42.789984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c3a4c-8683-4301-a73e-85c022631658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.790154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ff2fc-1451-45da-942d-c4a99a507df9 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:42.792791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c3a4c-8683-4301-a73e-85c022631658 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.796090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:42.797112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.798100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.798397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] PATH_INFO: `/v3/projects/b5c10c92633443d2bb77b2427d226e8a/users/a64d59fe9e554e3f83e52cc43c7597c7/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.806093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1336/2665] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:42.814003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5c10c92633443d2bb77b2427d226e8a, user_id=a64d59fe9e554e3f83e52cc43c7597c7, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:42.816811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:42.840772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1330/2666] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:42 2022] PUT /identity/v3/projects/b5c10c92633443d2bb77b2427d226e8a/users/a64d59fe9e554e3f83e52cc43c7597c7/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:42.853228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe1b86-dc9e-4691-a4c6-149a74b94298 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:42.853556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe1b86-dc9e-4691-a4c6-149a74b94298 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:42.854479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe1b86-dc9e-4691-a4c6-149a74b94298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:42.884185 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-11fe1b86-dc9e-4691-a4c6-149a74b94298 None None] MFA Rules not processed for user `a64d59fe9e554e3f83e52cc43c7597c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:42.885412 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-11fe1b86-dc9e-4691-a4c6-149a74b94298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:42.953914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1337/2667] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:42 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:42.966604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:42.968463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:43.030506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a20f30-3c74-4bf1-a7cf-a18ca272367e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.034248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a2399-8810-4977-b439-6f09d7a0adcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.034863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a2399-8810-4977-b439-6f09d7a0adcf service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.035132 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a2399-8810-4977-b439-6f09d7a0adcf service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.035424 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a2399-8810-4977-b439-6f09d7a0adcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.054698 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a2399-8810-4977-b439-6f09d7a0adcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.056772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a2399-8810-4977-b439-6f09d7a0adcf service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.067264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1331/2668] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:42 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:43.098869 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe1b86-dc9e-4691-a4c6-149a74b94298 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:43.100569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-11fe1b86-dc9e-4691-a4c6-149a74b94298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:43.136736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:43.177766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe1b86-dc9e-4691-a4c6-149a74b94298 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.182750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.183399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.183671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.183936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] PATH_INFO: `/v3/projects/af2e1681990b4f2dae32c0753bede855` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.185101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af2e1681990b4f2dae32c0753bede855)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.187094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.236178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af2e1681990b4f2dae32c0753bede855', 'request_id': 'req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:43.242015 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af2e1681990b4f2dae32c0753bede855', 'request_id': 'req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:43.248544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af2e1681990b4f2dae32c0753bede855', 'request_id': 'req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:43.265076 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] The token cache is being invalidate because project af2e1681990b4f2dae32c0753bede855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:44:43.265603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:43.266717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ed9c5bc7-b67e-49c0-a13a-eb5e318b1fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:44:43.268750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1338/2669] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:44:43 2022] DELETE /identity/v3/projects/af2e1681990b4f2dae32c0753bede855 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:43.274517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e2b8-03c1-4022-8b4f-302f1d43cfde None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.274808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e2b8-03c1-4022-8b4f-302f1d43cfde None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.275082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e2b8-03c1-4022-8b4f-302f1d43cfde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.297468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-a107e2b8-03c1-4022-8b4f-302f1d43cfde None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:43.298718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a107e2b8-03c1-4022-8b4f-302f1d43cfde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:43.358339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1332/2670] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:43.368384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a353e-1a22-4ef8-a161-315832edee1d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.368698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a353e-1a22-4ef8-a161-315832edee1d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.368981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a353e-1a22-4ef8-a161-315832edee1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.389495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-944a353e-1a22-4ef8-a161-315832edee1d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:43.390783 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-944a353e-1a22-4ef8-a161-315832edee1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:43.399379 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1339/2671] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:43.403721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107e2b8-03c1-4022-8b4f-302f1d43cfde None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:43.414037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107e2b8-03c1-4022-8b4f-302f1d43cfde None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.420989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.422107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.422397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.422736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.423487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:44:43.423915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.425819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.440045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1333/2672] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:44:43.444631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a353e-1a22-4ef8-a161-315832edee1d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:43.456899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a353e-1a22-4ef8-a161-315832edee1d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.468835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f44bb-c5cf-4015-8450-1f5baf0be9d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.469475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f44bb-c5cf-4015-8450-1f5baf0be9d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.469782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f44bb-c5cf-4015-8450-1f5baf0be9d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.470183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21f44bb-c5cf-4015-8450-1f5baf0be9d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.470871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f44bb-c5cf-4015-8450-1f5baf0be9d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.473659 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21f44bb-c5cf-4015-8450-1f5baf0be9d6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.529003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1340/2673] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:43 2022] POST /identity/v3/projects => generated 381 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:43.532866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:43.544706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe676dbf-0572-467b-a73b-50ae2f54501f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.614575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404010c0-b21f-48ee-917b-e1f7ddf72e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.614972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404010c0-b21f-48ee-917b-e1f7ddf72e63 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.615244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404010c0-b21f-48ee-917b-e1f7ddf72e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.615517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404010c0-b21f-48ee-917b-e1f7ddf72e63 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.616162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404010c0-b21f-48ee-917b-e1f7ddf72e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.618101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404010c0-b21f-48ee-917b-e1f7ddf72e63 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.635377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:43.645941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1334/2674] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:43 2022] POST /identity/v3/users => generated 291 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:43.647325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f44bb-c5cf-4015-8450-1f5baf0be9d6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:43.657880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21f44bb-c5cf-4015-8450-1f5baf0be9d6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.662630 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10444602-c7e0-4a9e-be8a-4a59ad672928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.663230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10444602-c7e0-4a9e-be8a-4a59ad672928 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.663504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10444602-c7e0-4a9e-be8a-4a59ad672928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.663767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10444602-c7e0-4a9e-be8a-4a59ad672928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.664401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10444602-c7e0-4a9e-be8a-4a59ad672928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.666028 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10444602-c7e0-4a9e-be8a-4a59ad672928 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.679418 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1341/2675] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:43.684036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404010c0-b21f-48ee-917b-e1f7ddf72e63 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:43.699537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404010c0-b21f-48ee-917b-e1f7ddf72e63 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.704286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89fcdfd-e5bf-4ba9-bb46-05452083faeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.704898 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fcdfd-e5bf-4ba9-bb46-05452083faeb admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.705177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fcdfd-e5bf-4ba9-bb46-05452083faeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.705447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89fcdfd-e5bf-4ba9-bb46-05452083faeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.706149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fcdfd-e5bf-4ba9-bb46-05452083faeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.707746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89fcdfd-e5bf-4ba9-bb46-05452083faeb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.718700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1335/2676] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:43.722919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10444602-c7e0-4a9e-be8a-4a59ad672928 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:43.732878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10444602-c7e0-4a9e-be8a-4a59ad672928 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.738206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6811f0-ac6d-47ac-97ed-f67ca2e8a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.738853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6811f0-ac6d-47ac-97ed-f67ca2e8a5fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.739109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6811f0-ac6d-47ac-97ed-f67ca2e8a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.739375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6811f0-ac6d-47ac-97ed-f67ca2e8a5fe admin admin] PATH_INFO: `/v3/projects/256be998e53944659093fec6649db429/users/63041ae2fa2a44b290199ce29efe905c/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.754066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6811f0-ac6d-47ac-97ed-f67ca2e8a5fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=256be998e53944659093fec6649db429, user_id=63041ae2fa2a44b290199ce29efe905c, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.756697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6811f0-ac6d-47ac-97ed-f67ca2e8a5fe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.771282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1342/2677] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:43 2022] PUT /identity/v3/projects/256be998e53944659093fec6649db429/users/63041ae2fa2a44b290199ce29efe905c/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:43.781488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042edbc5-ec3e-4179-a2c6-3e75e4c13dc3 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.781797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042edbc5-ec3e-4179-a2c6-3e75e4c13dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.782071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042edbc5-ec3e-4179-a2c6-3e75e4c13dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.810338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-042edbc5-ec3e-4179-a2c6-3e75e4c13dc3 None None] MFA Rules not processed for user `63041ae2fa2a44b290199ce29efe905c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:43.811583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-042edbc5-ec3e-4179-a2c6-3e75e4c13dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:43.879078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1336/2678] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:43.883717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6811f0-ac6d-47ac-97ed-f67ca2e8a5fe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:43.938316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:43.951337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6811f0-ac6d-47ac-97ed-f67ca2e8a5fe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:43.956487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807fcf7-1ea7-4879-92d7-146401ef3e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:43.957092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807fcf7-1ea7-4879-92d7-146401ef3e3d admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:43.957367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807fcf7-1ea7-4879-92d7-146401ef3e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:43.957605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807fcf7-1ea7-4879-92d7-146401ef3e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:43.958393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807fcf7-1ea7-4879-92d7-146401ef3e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:43.960322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807fcf7-1ea7-4879-92d7-146401ef3e3d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:43.988391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1343/2679] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:43 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:43.992688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042edbc5-ec3e-4179-a2c6-3e75e4c13dc3 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.004288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042edbc5-ec3e-4179-a2c6-3e75e4c13dc3 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.065238 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c3ef0-b540-4181-a016-d16cb5135965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.065880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c3ef0-b540-4181-a016-d16cb5135965 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.066154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c3ef0-b540-4181-a016-d16cb5135965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.066420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60c3ef0-b540-4181-a016-d16cb5135965 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.067052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c3ef0-b540-4181-a016-d16cb5135965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.068969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60c3ef0-b540-4181-a016-d16cb5135965 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.092735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1337/2680] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:43 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:44:44.097575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807fcf7-1ea7-4879-92d7-146401ef3e3d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.116362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807fcf7-1ea7-4879-92d7-146401ef3e3d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.122079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6c324-74d1-42ec-90ef-34a8cdafca7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.122713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6c324-74d1-42ec-90ef-34a8cdafca7e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.123115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6c324-74d1-42ec-90ef-34a8cdafca7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.123299 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6c324-74d1-42ec-90ef-34a8cdafca7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.123956 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6c324-74d1-42ec-90ef-34a8cdafca7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.125865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6c324-74d1-42ec-90ef-34a8cdafca7e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.136934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1344/2681] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:44.141679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c3ef0-b540-4181-a016-d16cb5135965 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.154916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60c3ef0-b540-4181-a016-d16cb5135965 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.159939 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47df28cd-aa25-45c8-a83e-e428dbf9dd22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.160582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df28cd-aa25-45c8-a83e-e428dbf9dd22 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.160876 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df28cd-aa25-45c8-a83e-e428dbf9dd22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.161146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47df28cd-aa25-45c8-a83e-e428dbf9dd22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.161820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df28cd-aa25-45c8-a83e-e428dbf9dd22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.163402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47df28cd-aa25-45c8-a83e-e428dbf9dd22 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.173783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1338/2682] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:44.177914 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6c324-74d1-42ec-90ef-34a8cdafca7e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.188089 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6c324-74d1-42ec-90ef-34a8cdafca7e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.193978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a4c5f-5ea5-401d-b0d3-fc41fef5cbbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.194631 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a4c5f-5ea5-401d-b0d3-fc41fef5cbbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.194902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a4c5f-5ea5-401d-b0d3-fc41fef5cbbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.195166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30a4c5f-5ea5-401d-b0d3-fc41fef5cbbd admin admin] PATH_INFO: `/v3/projects/45524b3a52e54064a2c0b892a5561973/users/dc23d04d25d245889ee439f4352d296f/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.210819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a4c5f-5ea5-401d-b0d3-fc41fef5cbbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=45524b3a52e54064a2c0b892a5561973, user_id=dc23d04d25d245889ee439f4352d296f, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.213480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30a4c5f-5ea5-401d-b0d3-fc41fef5cbbd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.231689 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1345/2683] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:44 2022] PUT /identity/v3/projects/45524b3a52e54064a2c0b892a5561973/users/dc23d04d25d245889ee439f4352d296f/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:44.248369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb4f4d-1a61-4d8f-b801-cdddc365f7f1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.248653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb4f4d-1a61-4d8f-b801-cdddc365f7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.248952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bb4f4d-1a61-4d8f-b801-cdddc365f7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.278798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d3bb4f4d-1a61-4d8f-b801-cdddc365f7f1 None None] MFA Rules not processed for user `dc23d04d25d245889ee439f4352d296f`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:44.280235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d3bb4f4d-1a61-4d8f-b801-cdddc365f7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:44.326475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:44.350201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1339/2684] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:44:44.355078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a4c5f-5ea5-401d-b0d3-fc41fef5cbbd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.420506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30a4c5f-5ea5-401d-b0d3-fc41fef5cbbd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.425775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fe3ee-5e24-42c4-9fa5-1c6e66465857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.426376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fe3ee-5e24-42c4-9fa5-1c6e66465857 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.426647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fe3ee-5e24-42c4-9fa5-1c6e66465857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.426919 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4fe3ee-5e24-42c4-9fa5-1c6e66465857 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.427528 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fe3ee-5e24-42c4-9fa5-1c6e66465857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.429455 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4fe3ee-5e24-42c4-9fa5-1c6e66465857 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.457331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1346/2685] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:44:44 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:44.461561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb4f4d-1a61-4d8f-b801-cdddc365f7f1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.473205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bb4f4d-1a61-4d8f-b801-cdddc365f7f1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.538699 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c440bc0-50e3-4a8c-8ce2-07da2569b654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.539259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c440bc0-50e3-4a8c-8ce2-07da2569b654 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.539530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c440bc0-50e3-4a8c-8ce2-07da2569b654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.539792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c440bc0-50e3-4a8c-8ce2-07da2569b654 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.540395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c440bc0-50e3-4a8c-8ce2-07da2569b654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.542374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c440bc0-50e3-4a8c-8ce2-07da2569b654 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.566570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1340/2686] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:44:44 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:44:44.570420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fe3ee-5e24-42c4-9fa5-1c6e66465857 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.581055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4fe3ee-5e24-42c4-9fa5-1c6e66465857 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.586426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211eece-78a9-42a9-948c-580b212ab26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.587206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211eece-78a9-42a9-948c-580b212ab26d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.587634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211eece-78a9-42a9-948c-580b212ab26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.587996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211eece-78a9-42a9-948c-580b212ab26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.588818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211eece-78a9-42a9-948c-580b212ab26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.591032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3211eece-78a9-42a9-948c-580b212ab26d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.602591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1347/2687] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:44.606790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c440bc0-50e3-4a8c-8ce2-07da2569b654 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.617220 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c440bc0-50e3-4a8c-8ce2-07da2569b654 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.622204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6036b-deb4-47c8-8962-8a8dd1a1a782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.622879 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6036b-deb4-47c8-8962-8a8dd1a1a782 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.623157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6036b-deb4-47c8-8962-8a8dd1a1a782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.623425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb6036b-deb4-47c8-8962-8a8dd1a1a782 admin admin] PATH_INFO: `/v3/projects/c4cd38aea3d54814962d5d52e767ff9f/users/82513b9544e048f6af8b072cc334af39/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.639201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6036b-deb4-47c8-8962-8a8dd1a1a782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cd38aea3d54814962d5d52e767ff9f, user_id=82513b9544e048f6af8b072cc334af39, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.641913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb6036b-deb4-47c8-8962-8a8dd1a1a782 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.657024 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1341/2688] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:44 2022] PUT /identity/v3/projects/c4cd38aea3d54814962d5d52e767ff9f/users/82513b9544e048f6af8b072cc334af39/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:44:44.661217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211eece-78a9-42a9-948c-580b212ab26d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.671354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211eece-78a9-42a9-948c-580b212ab26d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.733757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228e590d-b129-4fd7-86bc-022281c1600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.734514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e590d-b129-4fd7-86bc-022281c1600d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.734717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e590d-b129-4fd7-86bc-022281c1600d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.735045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228e590d-b129-4fd7-86bc-022281c1600d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.735709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e590d-b129-4fd7-86bc-022281c1600d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.737314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228e590d-b129-4fd7-86bc-022281c1600d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.747193 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1348/2689] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:44:44 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:44:44.751381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6036b-deb4-47c8-8962-8a8dd1a1a782 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.761950 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb6036b-deb4-47c8-8962-8a8dd1a1a782 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.769337 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.770084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.770691 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.770979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] PATH_INFO: `/v3/projects/c4cd38aea3d54814962d5d52e767ff9f/users/82513b9544e048f6af8b072cc334af39/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.773924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4cd38aea3d54814962d5d52e767ff9f, user_id=82513b9544e048f6af8b072cc334af39, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:44.776887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:44.789358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1342/2690] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:44:44 2022] PUT /identity/v3/projects/c4cd38aea3d54814962d5d52e767ff9f/users/82513b9544e048f6af8b072cc334af39/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:44:44.800952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4177b4-9fcf-4c0b-aceb-dd4c7f18a075 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.801216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4177b4-9fcf-4c0b-aceb-dd4c7f18a075 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.801469 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4177b4-9fcf-4c0b-aceb-dd4c7f18a075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:44.829481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:44.840225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-cb4177b4-9fcf-4c0b-aceb-dd4c7f18a075 None None] MFA Rules not processed for user `82513b9544e048f6af8b072cc334af39`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:44:44.841497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-cb4177b4-9fcf-4c0b-aceb-dd4c7f18a075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:44.911980 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1349/2691] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:44:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:44:44.925241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:44:44.928028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:44:44.992860 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cca39-5d99-47f5-84a8-a607415fc84e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:44:44.997080 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850675b-8c67-4bc6-9cae-bbd8af119ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:44:44.997759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850675b-8c67-4bc6-9cae-bbd8af119ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:44:44.998088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850675b-8c67-4bc6-9cae-bbd8af119ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:44:44.998415 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e850675b-8c67-4bc6-9cae-bbd8af119ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:44:45.049909 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:45.059577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850675b-8c67-4bc6-9cae-bbd8af119ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:44:45.061520 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e850675b-8c67-4bc6-9cae-bbd8af119ab5 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:44:45.070559 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1343/2692] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:44:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:44:54.907942 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=29 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:44:55.066669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:01.545022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4177b4-9fcf-4c0b-aceb-dd4c7f18a075 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:01.555158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4177b4-9fcf-4c0b-aceb-dd4c7f18a075 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:01.558925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:01.560886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:01.561157 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:01.561419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:01.620351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:01.621841 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:01.630742 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1350/2693] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:45:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:45:05.600000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850675b-8c67-4bc6-9cae-bbd8af119ab5 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:05.609937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e850675b-8c67-4bc6-9cae-bbd8af119ab5 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:05.615020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af68a29-e197-4cd8-a929-8f6264e52db5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:05.615877 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68a29-e197-4cd8-a929-8f6264e52db5 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:05.616205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68a29-e197-4cd8-a929-8f6264e52db5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:05.616594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af68a29-e197-4cd8-a929-8f6264e52db5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:05.631453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af68a29-e197-4cd8-a929-8f6264e52db5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:05.633580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af68a29-e197-4cd8-a929-8f6264e52db5 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:05.645507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1344/2694] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:45:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:45:06.873077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:06.874636 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:06.944181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb3960-8d58-484d-9332-75473ee99208 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:06.951065 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:06.951740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:06.952020 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:06.952290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] PATH_INFO: `/v3/users/a64d59fe9e554e3f83e52cc43c7597c7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:06.953502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64d59fe9e554e3f83e52cc43c7597c7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:06.955526 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:07.009316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64d59fe9e554e3f83e52cc43c7597c7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.018877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64d59fe9e554e3f83e52cc43c7597c7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.019782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64d59fe9e554e3f83e52cc43c7597c7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.028625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64d59fe9e554e3f83e52cc43c7597c7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.045908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64d59fe9e554e3f83e52cc43c7597c7'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.052576 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1351/2695] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:45:06 2022] DELETE /identity/v3/users/a64d59fe9e554e3f83e52cc43c7597c7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:45:07.057321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c82e1-255f-4c5e-84db-d8e12e5b945c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:07.057856 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c82e1-255f-4c5e-84db-d8e12e5b945c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:07.058170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c82e1-255f-4c5e-84db-d8e12e5b945c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:07.080032 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-cb5c82e1-255f-4c5e-84db-d8e12e5b945c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:45:07.081308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cb5c82e1-255f-4c5e-84db-d8e12e5b945c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:07.148408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1345/2696] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:45:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:45:07.159016 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:07.160508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:07.181897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:07.221986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c69dba-a7ae-4463-8638-c22169913e9b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:07.225801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:07.226445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:07.226721 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:07.226999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:07.236365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:07.237871 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:07.247287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1352/2697] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:45:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:45:07.790187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c82e1-255f-4c5e-84db-d8e12e5b945c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:07.791956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-cb5c82e1-255f-4c5e-84db-d8e12e5b945c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:07.806835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c82e1-255f-4c5e-84db-d8e12e5b945c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:07.812339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:07.813124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:07.813579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:07.813891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] PATH_INFO: `/v3/projects/b5c10c92633443d2bb77b2427d226e8a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:07.815166 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5c10c92633443d2bb77b2427d226e8a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:07.817094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:07.868115 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5c10c92633443d2bb77b2427d226e8a', 'request_id': 'req-39a4e71c-5365-4150-b637-472119254ca0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.874799 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5c10c92633443d2bb77b2427d226e8a', 'request_id': 'req-39a4e71c-5365-4150-b637-472119254ca0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.880949 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5c10c92633443d2bb77b2427d226e8a', 'request_id': 'req-39a4e71c-5365-4150-b637-472119254ca0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.897683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] The token cache is being invalidate because project b5c10c92633443d2bb77b2427d226e8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:45:07.897968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.898672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-39a4e71c-5365-4150-b637-472119254ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:07.900462 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1346/2698] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:45:07 2022] DELETE /identity/v3/projects/b5c10c92633443d2bb77b2427d226e8a => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:45:09.631064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:09.632666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:09.646409 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a079933-baf1-4d55-a215-6abca1034fb6 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:09.718864 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:09.719515 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:09.719791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:09.720063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] PATH_INFO: `/v3/users/63041ae2fa2a44b290199ce29efe905c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:09.721352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=63041ae2fa2a44b290199ce29efe905c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:09.723347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:09.788429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63041ae2fa2a44b290199ce29efe905c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:09.795655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63041ae2fa2a44b290199ce29efe905c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:09.796356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63041ae2fa2a44b290199ce29efe905c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:09.806091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63041ae2fa2a44b290199ce29efe905c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:09.818344 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63041ae2fa2a44b290199ce29efe905c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:09.824491 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1353/2699] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:45:09 2022] DELETE /identity/v3/users/63041ae2fa2a44b290199ce29efe905c => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:45:09.829339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65976b-e4b4-45f7-8c89-f4f2fc0f16a0 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:09.829931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65976b-e4b4-45f7-8c89-f4f2fc0f16a0 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:09.829931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65976b-e4b4-45f7-8c89-f4f2fc0f16a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:09.852944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ac65976b-e4b4-45f7-8c89-f4f2fc0f16a0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:45:09.854377 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ac65976b-e4b4-45f7-8c89-f4f2fc0f16a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:09.919028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1347/2700] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:45:09.929827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:09.931391 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:09.976872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:09.995026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e1f8f9-bf21-4e62-ae3f-9215a32bdf5d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:09.999139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c6f91-1b7d-43a3-ba2b-8fc208eca8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:09.999759 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6f91-1b7d-43a3-ba2b-8fc208eca8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:10.000037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6f91-1b7d-43a3-ba2b-8fc208eca8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:10.000310 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c6f91-1b7d-43a3-ba2b-8fc208eca8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:10.012142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c6f91-1b7d-43a3-ba2b-8fc208eca8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:10.013607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7c6f91-1b7d-43a3-ba2b-8fc208eca8f1 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:10.024380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1354/2701] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:45:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:45:10.308765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65976b-e4b4-45f7-8c89-f4f2fc0f16a0 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:10.310345 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ac65976b-e4b4-45f7-8c89-f4f2fc0f16a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:10.323702 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65976b-e4b4-45f7-8c89-f4f2fc0f16a0 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:10.338248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:10.339422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:10.339662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:10.340005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] PATH_INFO: `/v3/projects/256be998e53944659093fec6649db429` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:10.341100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=256be998e53944659093fec6649db429)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:10.343200 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:10.350598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:10.395308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '256be998e53944659093fec6649db429', 'request_id': 'req-2177afb6-4840-4398-bdac-fd5c2ab3e423'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.401130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '256be998e53944659093fec6649db429', 'request_id': 'req-2177afb6-4840-4398-bdac-fd5c2ab3e423'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.407872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '256be998e53944659093fec6649db429', 'request_id': 'req-2177afb6-4840-4398-bdac-fd5c2ab3e423'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.426168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] The token cache is being invalidate because project 256be998e53944659093fec6649db429 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:45:10.426532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.427490 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.429812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1348/2702] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:45:10 2022] DELETE /identity/v3/projects/256be998e53944659093fec6649db429 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:45:10.435199 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c6f91-1b7d-43a3-ba2b-8fc208eca8f1 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:10.447951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c6f91-1b7d-43a3-ba2b-8fc208eca8f1 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:10.512917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:10.513387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:10.513695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:10.513960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] PATH_INFO: `/v3/users/dc23d04d25d245889ee439f4352d296f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:10.515110 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc23d04d25d245889ee439f4352d296f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:10.517063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:10.568503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc23d04d25d245889ee439f4352d296f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.575446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc23d04d25d245889ee439f4352d296f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.576087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc23d04d25d245889ee439f4352d296f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.585673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc23d04d25d245889ee439f4352d296f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.599327 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc23d04d25d245889ee439f4352d296f'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:10.607012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1355/2703] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:45:10 2022] DELETE /identity/v3/users/dc23d04d25d245889ee439f4352d296f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:45:10.888176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:10.889928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:10.909722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2177afb6-4840-4398-bdac-fd5c2ab3e423 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:10.968715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:10.969354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:10.969645 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:10.969929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] PATH_INFO: `/v3/projects/45524b3a52e54064a2c0b892a5561973` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:10.971087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] RBAC: Authorizing `identity:delete_project(project_id=45524b3a52e54064a2c0b892a5561973)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:10.972997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:11.021286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45524b3a52e54064a2c0b892a5561973', 'request_id': 'req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.027768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45524b3a52e54064a2c0b892a5561973', 'request_id': 'req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.034382 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45524b3a52e54064a2c0b892a5561973', 'request_id': 'req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.048902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] The token cache is being invalidate because project 45524b3a52e54064a2c0b892a5561973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:45:11.049192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.049859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.051494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1349/2704] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:45:10 2022] DELETE /identity/v3/projects/45524b3a52e54064a2c0b892a5561973 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:45:11.055577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:11.057273 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:11.070916 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18959a50-bae8-460e-8c1d-9601e5dc35cd admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:11.127725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:11.128402 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:11.128678 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:11.128977 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] PATH_INFO: `/v3/users/82513b9544e048f6af8b072cc334af39` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:11.130172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] RBAC: Authorizing `identity:delete_user(user_id=82513b9544e048f6af8b072cc334af39)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:11.132204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:11.134739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:11.190826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82513b9544e048f6af8b072cc334af39'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.198441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82513b9544e048f6af8b072cc334af39'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.199194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82513b9544e048f6af8b072cc334af39'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.208653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82513b9544e048f6af8b072cc334af39'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.222075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-cd6a132f-a9e0-43a3-b78f-5e1618df0dff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82513b9544e048f6af8b072cc334af39'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.228159 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1356/2705] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:45:11 2022] DELETE /identity/v3/users/82513b9544e048f6af8b072cc334af39 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:45:11.522135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:11.523810 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:11.537502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0396a8a7-3a2a-49b3-93b3-a74dd9bd346a admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:11.560064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:11.597874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:11.598529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:11.598804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:11.599078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] PATH_INFO: `/v3/projects/c4cd38aea3d54814962d5d52e767ff9f` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:11.600186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4cd38aea3d54814962d5d52e767ff9f)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:11.602248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:11.652835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4cd38aea3d54814962d5d52e767ff9f', 'request_id': 'req-a3e782f4-d213-40dd-aac8-db9308469534'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.658614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4cd38aea3d54814962d5d52e767ff9f', 'request_id': 'req-a3e782f4-d213-40dd-aac8-db9308469534'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.665409 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4cd38aea3d54814962d5d52e767ff9f', 'request_id': 'req-a3e782f4-d213-40dd-aac8-db9308469534'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.682760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] The token cache is being invalidate because project c4cd38aea3d54814962d5d52e767ff9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:45:11.683046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.683660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a3e782f4-d213-40dd-aac8-db9308469534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:45:11.685155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1350/2706] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:45:11 2022] DELETE /identity/v3/projects/c4cd38aea3d54814962d5d52e767ff9f => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:45:11.691706 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6ea6-bd4a-4177-9068-e33b7a9c3237 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:11.691999 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6ea6-bd4a-4177-9068-e33b7a9c3237 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:11.692272 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e6ea6-bd4a-4177-9068-e33b7a9c3237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:11.716006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-eb1e6ea6-bd4a-4177-9068-e33b7a9c3237 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:45:11.717268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eb1e6ea6-bd4a-4177-9068-e33b7a9c3237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:11.786295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1357/2707] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:45:11.800186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b619e08-82b4-4bb9-8541-1a5dd9334989 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:11.800743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b619e08-82b4-4bb9-8541-1a5dd9334989 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:11.801146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b619e08-82b4-4bb9-8541-1a5dd9334989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:11.828296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-9b619e08-82b4-4bb9-8541-1a5dd9334989 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:45:11.829806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9b619e08-82b4-4bb9-8541-1a5dd9334989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:11.839867 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1351/2708] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:45:11.844910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e6ea6-bd4a-4177-9068-e33b7a9c3237 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:11.856852 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e6ea6-bd4a-4177-9068-e33b7a9c3237 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:11.863332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:11.864712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:11.865155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:11.865610 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:11.866604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:45:11.867047 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:11.869685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:11.883651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1358/2709] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:45:11.890540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b619e08-82b4-4bb9-8541-1a5dd9334989 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:11.904594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b619e08-82b4-4bb9-8541-1a5dd9334989 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:11.911067 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3418129-e861-4f66-80ae-84c24dba13f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:11.911695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3418129-e861-4f66-80ae-84c24dba13f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:11.911999 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3418129-e861-4f66-80ae-84c24dba13f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:11.912258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3418129-e861-4f66-80ae-84c24dba13f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:11.913004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3418129-e861-4f66-80ae-84c24dba13f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:11.915030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3418129-e861-4f66-80ae-84c24dba13f5 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:11.947051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1352/2710] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:45:11 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:45:11.951643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:11.962314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0f6e3-62c0-440b-973d-45c0f0089544 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:12.023906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e854-1c0f-4108-9960-68b9859508c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:12.024544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e854-1c0f-4108-9960-68b9859508c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:12.024825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e854-1c0f-4108-9960-68b9859508c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:12.025106 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209e854-1c0f-4108-9960-68b9859508c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:12.025775 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209e854-1c0f-4108-9960-68b9859508c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:12.027732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209e854-1c0f-4108-9960-68b9859508c7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:12.046265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:12.060229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1359/2711] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:45:11 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:45:12.061917 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3418129-e861-4f66-80ae-84c24dba13f5 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:12.073412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3418129-e861-4f66-80ae-84c24dba13f5 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:12.078714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa032-ed5c-4966-b484-569f28b66672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:12.079362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa032-ed5c-4966-b484-569f28b66672 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:12.079631 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa032-ed5c-4966-b484-569f28b66672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:12.079897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8aa032-ed5c-4966-b484-569f28b66672 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:12.080583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8aa032-ed5c-4966-b484-569f28b66672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:12.082286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8aa032-ed5c-4966-b484-569f28b66672 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:12.095683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1353/2712] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:45:12.097991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e854-1c0f-4108-9960-68b9859508c7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:12.108911 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209e854-1c0f-4108-9960-68b9859508c7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:12.114109 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b431f2-ef67-401a-812a-a52c5b03b246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:12.114729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b431f2-ef67-401a-812a-a52c5b03b246 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:12.115004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b431f2-ef67-401a-812a-a52c5b03b246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:12.115316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b431f2-ef67-401a-812a-a52c5b03b246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:12.115950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b431f2-ef67-401a-812a-a52c5b03b246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:12.117803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b431f2-ef67-401a-812a-a52c5b03b246 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:12.132920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1360/2713] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:45:12.134184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa032-ed5c-4966-b484-569f28b66672 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:12.145077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8aa032-ed5c-4966-b484-569f28b66672 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:12.149562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:12.150260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:12.150532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:12.150798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] PATH_INFO: `/v3/projects/5591c836050d488487149a6f548393c9/users/e1596db886c046aaa465c56b838051eb/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:12.165941 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5591c836050d488487149a6f548393c9, user_id=e1596db886c046aaa465c56b838051eb, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:12.168493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:12.184336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1354/2714] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:45:12 2022] PUT /identity/v3/projects/5591c836050d488487149a6f548393c9/users/e1596db886c046aaa465c56b838051eb/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:45:12.190093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b044-cf67-4d9e-bfa8-9795d4330266 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:12.190530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b044-cf67-4d9e-bfa8-9795d4330266 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:12.190937 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e10b044-cf67-4d9e-bfa8-9795d4330266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:12.211993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6e10b044-cf67-4d9e-bfa8-9795d4330266 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:45:12.213376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6e10b044-cf67-4d9e-bfa8-9795d4330266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:12.286352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1361/2715] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:45:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:45:12.296937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:12.299780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:12.370392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f71814-c657-4c2e-96d9-69184e845aa7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:12.374500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:12.375177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:12.375453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:12.375726 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:12.382121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:12.385857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:12.387655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:12.398560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1355/2716] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:45:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:45:13.049944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d05ae-e448-4d6a-aa1c-fb7160f3351f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:13.050244 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d05ae-e448-4d6a-aa1c-fb7160f3351f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:13.050513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d05ae-e448-4d6a-aa1c-fb7160f3351f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:13.079820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-dd6d05ae-e448-4d6a-aa1c-fb7160f3351f None None] MFA Rules not processed for user `e1596db886c046aaa465c56b838051eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:45:13.081035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-dd6d05ae-e448-4d6a-aa1c-fb7160f3351f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:13.156421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1362/2717] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:45:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:45:13.168891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:13.170598 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:13.244843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2e21b8-c3a3-480d-800f-416c88df76f5 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:13.248539 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:13.249130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:13.249412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:13.249820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:13.258062 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:13.259540 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:13.270514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1356/2718] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:45:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:45:14.659321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424cef7-8516-4fd4-9a55-08657c28ea16 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:14.659857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424cef7-8516-4fd4-9a55-08657c28ea16 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:14.659857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424cef7-8516-4fd4-9a55-08657c28ea16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:14.682934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8424cef7-8516-4fd4-9a55-08657c28ea16 None None] MFA Rules not processed for user `e1596db886c046aaa465c56b838051eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:45:14.684160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8424cef7-8516-4fd4-9a55-08657c28ea16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:14.692369 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1363/2719] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:45:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:45:23.264964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=29 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:24.678019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=68 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:45:49.356704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:45:49.358289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:49.431431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6394-02b3-40fe-8d4e-bf753b1c76df service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:45:49.436796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:45:49.437430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:45:49.437740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:45:49.438010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:45:49.439869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:45:49.451531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:45:49.453114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:45:49.463047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1357/2720] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:45:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:45:59.459084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=32 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:46:14.772449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7790bba-c9f6-452f-9223-0a8bb6166389 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:46:14.773093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7790bba-c9f6-452f-9223-0a8bb6166389 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:46:14.773093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7790bba-c9f6-452f-9223-0a8bb6166389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:46:14.793521 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e7790bba-c9f6-452f-9223-0a8bb6166389 None None] MFA Rules not processed for user `e1596db886c046aaa465c56b838051eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:46:14.794786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e7790bba-c9f6-452f-9223-0a8bb6166389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:46:14.803099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1364/2721] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:46:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:46:24.788606 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:46:58.424047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:46:58.434227 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1bb799-6440-4989-ad2f-086dfeefee5a service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:46:58.438797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:46:58.439396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:46:58.439671 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:46:58.439928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:46:58.441986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:46:58.526770 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:46:58.528212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:46:58.546945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1358/2722] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:46:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:47:08.544256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=41 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:47:21.763101 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205305-aafb-42b3-82fd-0a5463884efc None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:47:21.763614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205305-aafb-42b3-82fd-0a5463884efc None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:47:21.763713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48205305-aafb-42b3-82fd-0a5463884efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:47:21.783684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-48205305-aafb-42b3-82fd-0a5463884efc None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:47:21.784807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-48205305-aafb-42b3-82fd-0a5463884efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:47:21.843748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1365/2723] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:47:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:47:21.892365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:47:21.894307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:47:21.966887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81708a9e-0137-4954-9611-add1da9900fa service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:47:21.977097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894968a4-5c3b-409f-b91e-ae70618f8c69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:47:21.978082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894968a4-5c3b-409f-b91e-ae70618f8c69 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:47:21.978359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894968a4-5c3b-409f-b91e-ae70618f8c69 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:47:21.978627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894968a4-5c3b-409f-b91e-ae70618f8c69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:47:21.987440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894968a4-5c3b-409f-b91e-ae70618f8c69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:47:21.989008 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894968a4-5c3b-409f-b91e-ae70618f8c69 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:47:21.997078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1359/2724] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:47:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 05 14:47:31.835393 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=25 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:47:31.993924 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=32 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:47:38.513266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb64146-614c-48ae-b1e8-035a9eacbc0d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:47:38.513844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb64146-614c-48ae-b1e8-035a9eacbc0d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:47:38.513844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb64146-614c-48ae-b1e8-035a9eacbc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:47:38.534289 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4eb64146-614c-48ae-b1e8-035a9eacbc0d None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:47:38.535513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4eb64146-614c-48ae-b1e8-035a9eacbc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:47:38.543909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1366/2725] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 14:47:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:47:38.556886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894968a4-5c3b-409f-b91e-ae70618f8c69 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:47:38.568447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894968a4-5c3b-409f-b91e-ae70618f8c69 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:47:38.575430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f595ef-5017-4cc6-9ddb-59641142f9eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:47:38.576046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f595ef-5017-4cc6-9ddb-59641142f9eb service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:47:38.576325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f595ef-5017-4cc6-9ddb-59641142f9eb service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:47:38.576613 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f595ef-5017-4cc6-9ddb-59641142f9eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:47:38.584890 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f595ef-5017-4cc6-9ddb-59641142f9eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:47:38.586873 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f595ef-5017-4cc6-9ddb-59641142f9eb service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:47:38.594819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1360/2726] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:47:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:47:40.600693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb64146-614c-48ae-b1e8-035a9eacbc0d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:47:40.632591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb64146-614c-48ae-b1e8-035a9eacbc0d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:47:40.638046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:47:40.638719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:47:40.639045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:47:40.639308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] PATH_INFO: `/v3/users/e1596db886c046aaa465c56b838051eb` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:47:40.640699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1596db886c046aaa465c56b838051eb)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:47:40.642821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:47:40.694776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1596db886c046aaa465c56b838051eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:40.701803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1596db886c046aaa465c56b838051eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:40.702619 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1596db886c046aaa465c56b838051eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:40.715749 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1596db886c046aaa465c56b838051eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:40.728514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1596db886c046aaa465c56b838051eb'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:40.734910 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1367/2727] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:47:40 2022] DELETE /identity/v3/users/e1596db886c046aaa465c56b838051eb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:47:40.939758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f595ef-5017-4cc6-9ddb-59641142f9eb service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:47:40.951392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f595ef-5017-4cc6-9ddb-59641142f9eb service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:47:41.015053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:47:41.015866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:47:41.016135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:47:41.016597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] PATH_INFO: `/v3/projects/5591c836050d488487149a6f548393c9` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:47:41.018579 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5591c836050d488487149a6f548393c9)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:47:41.020638 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:47:41.077316 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5591c836050d488487149a6f548393c9', 'request_id': 'req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:41.083131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5591c836050d488487149a6f548393c9', 'request_id': 'req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:41.091015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5591c836050d488487149a6f548393c9', 'request_id': 'req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:41.121312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] The token cache is being invalidate because project 5591c836050d488487149a6f548393c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:47:41.121594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:41.122339 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:47:41.123962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1361/2728] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:47:40 2022] DELETE /identity/v3/projects/5591c836050d488487149a6f548393c9 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:47:50.731377 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:47:51.118321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:48:43.060396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:48:43.062032 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:48:43.123646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da67b48b-45a2-4de4-8fd7-290153884efe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:48:43.175266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:48:43.175891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:48:43.176170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:48:43.176436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:48:43.178367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:48:43.187994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:48:43.189478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:48:43.197628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1368/2729] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:48:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:48:53.194213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=46 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:50:57.301065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:50:57.302572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:50:57.455310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc7b69-1b4a-422a-b0aa-5bbac7845cc0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:50:57.461425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:50:57.462070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:50:57.462342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:50:57.462617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:50:57.464660 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:50:57.476100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:50:57.477670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:50:57.487313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1362/2730] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:50:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:51:07.483570 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=63 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:53:48.000811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:53:48.002513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:53:48.078866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee76bdb-0599-4d55-a5cd-a63524d5532c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:53:48.084059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc79781-76d9-4c2b-a4bc-3fde424a24a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:53:48.084668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc79781-76d9-4c2b-a4bc-3fde424a24a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:53:48.084968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc79781-76d9-4c2b-a4bc-3fde424a24a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:53:48.085231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc79781-76d9-4c2b-a4bc-3fde424a24a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:53:48.094475 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc79781-76d9-4c2b-a4bc-3fde424a24a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:53:48.096593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc79781-76d9-4c2b-a4bc-3fde424a24a5 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:53:48.108410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1369/2731] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:53:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:53:58.103547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=28 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:55:42.110391 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:42.112075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:42.200855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d7e77-7b1c-4b5f-bc6f-16f21c2d4b91 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:42.205707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417b661f770846f189c36d0804b6dce5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:42.206321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:42.206596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:42.206865 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:42.209050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:42.297526 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:42.299001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:42.317496 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1363/2732] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:55:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:55:42.468230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a4a99-deea-408a-8c8b-941eed1c8cb4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:42.468751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a4a99-deea-408a-8c8b-941eed1c8cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:42.468876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a4a99-deea-408a-8c8b-941eed1c8cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:42.533328 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1b3a4a99-deea-408a-8c8b-941eed1c8cb4 None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:55:42.535184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1b3a4a99-deea-408a-8c8b-941eed1c8cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:42.617350 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1370/2733] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:55:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:55:42.675466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:42.677179 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:42.749183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:55:42.774208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ed387-a9bb-4445-bff2-622230d4590d service glance] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:42.779374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dcf1a-45ca-409c-bae8-739c6a7abb55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:42.779984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dcf1a-45ca-409c-bae8-739c6a7abb55 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:42.780260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dcf1a-45ca-409c-bae8-739c6a7abb55 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:42.780530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7dcf1a-45ca-409c-bae8-739c6a7abb55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:42.789147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7dcf1a-45ca-409c-bae8-739c6a7abb55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:42.790750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7dcf1a-45ca-409c-bae8-739c6a7abb55 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:42.798615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1364/2734] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:55:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:55:43.342229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296dcb4-7e16-4dca-9ad2-bce24ba4a7e4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:43.342509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296dcb4-7e16-4dca-9ad2-bce24ba4a7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:43.342779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7296dcb4-7e16-4dca-9ad2-bce24ba4a7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:43.362736 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7296dcb4-7e16-4dca-9ad2-bce24ba4a7e4 None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:55:43.363982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7296dcb4-7e16-4dca-9ad2-bce24ba4a7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:43.372348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1371/2735] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 14:55:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:55:43.384714 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dcf1a-45ca-409c-bae8-739c6a7abb55 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:43.395967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7dcf1a-45ca-409c-bae8-739c6a7abb55 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:43.401435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889afd65-c4f1-4a03-b73f-9084e765fa9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:43.402088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889afd65-c4f1-4a03-b73f-9084e765fa9f service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:43.402367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889afd65-c4f1-4a03-b73f-9084e765fa9f service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:43.402641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889afd65-c4f1-4a03-b73f-9084e765fa9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:43.410772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889afd65-c4f1-4a03-b73f-9084e765fa9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:43.412430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889afd65-c4f1-4a03-b73f-9084e765fa9f service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:43.420493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1365/2736] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:55:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:55:43.671979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c9035-12a4-496c-b15d-351516a4aa01 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:43.672456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c9035-12a4-496c-b15d-351516a4aa01 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:43.672558 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545c9035-12a4-496c-b15d-351516a4aa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:43.692564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-545c9035-12a4-496c-b15d-351516a4aa01 None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:55:43.693836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-545c9035-12a4-496c-b15d-351516a4aa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:43.701297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1372/2737] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:55:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:55:43.713662 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889afd65-c4f1-4a03-b73f-9084e765fa9f service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:43.726700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889afd65-c4f1-4a03-b73f-9084e765fa9f service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:43.735152 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173458cf-e397-41f0-945a-578472d5a535 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:43.735804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173458cf-e397-41f0-945a-578472d5a535 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:43.736084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173458cf-e397-41f0-945a-578472d5a535 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:43.736350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173458cf-e397-41f0-945a-578472d5a535 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:43.749281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173458cf-e397-41f0-945a-578472d5a535 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:43.750907 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173458cf-e397-41f0-945a-578472d5a535 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:43.761681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1366/2738] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:55:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:55:44.116288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2189b-8ee8-406e-8347-45a9c3de3853 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:44.116566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2189b-8ee8-406e-8347-45a9c3de3853 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:44.116831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2189b-8ee8-406e-8347-45a9c3de3853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:44.140595 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-73c2189b-8ee8-406e-8347-45a9c3de3853 None None] MFA Rules not processed for user `8f18740d784843ada83018867899429a`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:55:44.142339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73c2189b-8ee8-406e-8347-45a9c3de3853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:44.154707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1373/2739] 10.222.0.23 () {58 vars in 967 bytes} [Wed Oct 5 14:55:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:55:44.167683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173458cf-e397-41f0-945a-578472d5a535 service swift] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:44.178806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173458cf-e397-41f0-945a-578472d5a535 service swift] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:44.185310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1103e-732c-481e-bfd4-b21eaf805078 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01286f6f65b44519b645927f6c70ec5b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:44.185968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1103e-732c-481e-bfd4-b21eaf805078 service swift] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:44.186233 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1103e-732c-481e-bfd4-b21eaf805078 service swift] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:44.186500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1103e-732c-481e-bfd4-b21eaf805078 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:44.195351 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1103e-732c-481e-bfd4-b21eaf805078 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:44.196959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a1103e-732c-481e-bfd4-b21eaf805078 service swift] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:44.205751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1367/2740] 10.222.0.23 () {60 vars in 1384 bytes} [Wed Oct 5 14:55:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 05 14:55:46.342836 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2189b-8ee8-406e-8347-45a9c3de3853 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:46.345394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-73c2189b-8ee8-406e-8347-45a9c3de3853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:46.414697 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2189b-8ee8-406e-8347-45a9c3de3853 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:46.419778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:46.420431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:46.420710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:46.420968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] PATH_INFO: `/v3/users/005d0f9318534cfc98c37d822e49bdc4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:46.422230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=005d0f9318534cfc98c37d822e49bdc4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:46.424162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:46.469894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:55:46.476278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '005d0f9318534cfc98c37d822e49bdc4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.483658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '005d0f9318534cfc98c37d822e49bdc4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.484316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '005d0f9318534cfc98c37d822e49bdc4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.493367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '005d0f9318534cfc98c37d822e49bdc4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.505383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '005d0f9318534cfc98c37d822e49bdc4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.511495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1374/2741] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:55:46 2022] DELETE /identity/v3/users/005d0f9318534cfc98c37d822e49bdc4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:55:46.516280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47285e4d-ab8d-465d-9363-63f8c216e44a None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:46.516784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47285e4d-ab8d-465d-9363-63f8c216e44a None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:46.516784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47285e4d-ab8d-465d-9363-63f8c216e44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:46.544902 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-47285e4d-ab8d-465d-9363-63f8c216e44a None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:55:46.546141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47285e4d-ab8d-465d-9363-63f8c216e44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:46.604275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1368/2742] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:55:46.614401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:46.616094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:46.682421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf5946-977e-4c4a-ad6d-da5e3ab52e77 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:46.687589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:46.688206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:46.688479 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:46.688744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:46.701115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:46.702637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:46.718413 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1375/2743] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:55:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:55:46.896795 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47285e4d-ab8d-465d-9363-63f8c216e44a None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:46.898425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-47285e4d-ab8d-465d-9363-63f8c216e44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:46.910853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47285e4d-ab8d-465d-9363-63f8c216e44a None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:46.916506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:46.917142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:46.917418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:46.917743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] PATH_INFO: `/v3/projects/bea771b3ff124fc8b9b9ac57f8ea12b4` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:46.918885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bea771b3ff124fc8b9b9ac57f8ea12b4)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:46.920778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:46.969784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'request_id': 'req-61ff3295-27f9-4185-9797-e145269f204e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.975330 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'request_id': 'req-61ff3295-27f9-4185-9797-e145269f204e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.981681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bea771b3ff124fc8b9b9ac57f8ea12b4', 'request_id': 'req-61ff3295-27f9-4185-9797-e145269f204e'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.996403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] The token cache is being invalidate because project bea771b3ff124fc8b9b9ac57f8ea12b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:55:46.996695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.997425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:55:46.998916 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1369/2744] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:55:46 2022] DELETE /identity/v3/projects/bea771b3ff124fc8b9b9ac57f8ea12b4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:55:56.714347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=5,keystone:SELECT=40,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:55:56.993403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:55:58.939763 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:55:58.941342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:58.956448 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e69bb-564c-4c66-b83a-853f9eeac02b service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:55:59.008278 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:55:59.008886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:55:59.009162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:55:59.009424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:55:59.011560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:55:59.074329 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:55:59.075774 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:55:59.085810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1376/2745] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:56:01.285811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:01.287379 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:01.374053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff3295-27f9-4185-9797-e145269f204e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:01.379247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:01.379874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:01.380147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:01.380410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:01.382174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:01.394498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:01.396113 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:01.407978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1370/2746] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:56:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:56:02.225337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:02.240348 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724dc462-2988-4084-bc69-b07f69e95a7c service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:02.247099 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:02.247746 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:02.248019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:02.248285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:02.250404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:02.263486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:02.265122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:02.276676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1377/2747] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:56:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:56:05.259165 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:05.260923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:05.278715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12ca73-0c71-4fa2-8498-0eb65fd30333 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:05.286734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:05.287556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:05.287899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:05.288234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] PATH_INFO: `/v3/users/7859215cd50c4a4f84e27788457dfef6` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:05.303729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7859215cd50c4a4f84e27788457dfef6)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:05.305771 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:05.356364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7859215cd50c4a4f84e27788457dfef6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.363205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7859215cd50c4a4f84e27788457dfef6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.363864 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7859215cd50c4a4f84e27788457dfef6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.372874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7859215cd50c4a4f84e27788457dfef6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.385222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7859215cd50c4a4f84e27788457dfef6'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.391503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1371/2748] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:56:05 2022] DELETE /identity/v3/users/7859215cd50c4a4f84e27788457dfef6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:56:05.589341 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:05.603882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d275e910-eafe-4306-8680-d95a27e2260d service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:05.665288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:05.665959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:05.666265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:05.666529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] PATH_INFO: `/v3/projects/31137b9776dd4dbf89dcc7b10ef4e65b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:05.677290 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31137b9776dd4dbf89dcc7b10ef4e65b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:05.679247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:05.714129 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:56:05.728252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31137b9776dd4dbf89dcc7b10ef4e65b', 'request_id': 'req-87e1c67c-53f7-414d-b21e-4f9ef40df039'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.733587 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31137b9776dd4dbf89dcc7b10ef4e65b', 'request_id': 'req-87e1c67c-53f7-414d-b21e-4f9ef40df039'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.740102 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31137b9776dd4dbf89dcc7b10ef4e65b', 'request_id': 'req-87e1c67c-53f7-414d-b21e-4f9ef40df039'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.755580 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] The token cache is being invalidate because project 31137b9776dd4dbf89dcc7b10ef4e65b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:56:05.755921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.756585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.758352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1378/2749] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:56:05 2022] DELETE /identity/v3/projects/31137b9776dd4dbf89dcc7b10ef4e65b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:56:05.763241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:05.764893 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:05.778848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd9d5e-fe90-441d-b11a-82fd00590fa7 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:05.833177 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:56:05.845931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:05.846562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:05.846815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:05.847107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] PATH_INFO: `/v3/users/ceb2c96376ec456b83651cbe48a2590d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:05.861517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ceb2c96376ec456b83651cbe48a2590d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:05.863508 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:05.911031 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ceb2c96376ec456b83651cbe48a2590d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.917751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ceb2c96376ec456b83651cbe48a2590d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.918384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ceb2c96376ec456b83651cbe48a2590d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.927534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ceb2c96376ec456b83651cbe48a2590d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.939260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ceb2c96376ec456b83651cbe48a2590d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:05.945049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1372/2750] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:56:05 2022] DELETE /identity/v3/users/ceb2c96376ec456b83651cbe48a2590d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:56:06.135626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:06.137180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:06.152117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e1c67c-53f7-414d-b21e-4f9ef40df039 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:06.217192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:06.217881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:06.218154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:06.218405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] PATH_INFO: `/v3/projects/3a11431ac93b43948e378382b9018d9d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:06.229792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a11431ac93b43948e378382b9018d9d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:06.231731 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:06.286839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a11431ac93b43948e378382b9018d9d', 'request_id': 'req-776b695a-4c97-4243-b9c1-6cf22566aa54'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:06.294523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a11431ac93b43948e378382b9018d9d', 'request_id': 'req-776b695a-4c97-4243-b9c1-6cf22566aa54'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:06.301179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a11431ac93b43948e378382b9018d9d', 'request_id': 'req-776b695a-4c97-4243-b9c1-6cf22566aa54'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:06.315740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] The token cache is being invalidate because project 3a11431ac93b43948e378382b9018d9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:56:06.316031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:06.316675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:06.318227 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1379/2751] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:56:06 2022] DELETE /identity/v3/projects/3a11431ac93b43948e378382b9018d9d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:56:11.342617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:11.345225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:11.442874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb666ce-2c14-43bd-8eb5-dbacb02a4ddd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:11.501059 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:11.501948 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:11.502407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:11.502790 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:11.505683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:11.515531 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:56:11.613500 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:11.614641 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:11.634515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1373/2752] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:56:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:56:16.313100 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=68,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:56:21.635199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=36 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:56:42.813677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:42.815234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:42.872952 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776b695a-4c97-4243-b9c1-6cf22566aa54 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:42.876791 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:42.877399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:42.877663 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:42.877979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:42.879801 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:42.943211 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:42.944641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:42.954362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1380/2753] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:56:45.780649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:45.782156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:45.903400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf63ec-4651-47f2-8955-6be42d737579 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:45.910297 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:45.911190 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:45.911541 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:45.911883 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:45.914001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:45.933444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:45.935585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:45.954528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1374/2754] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:56:45.974178 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:45.975950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:45.991263 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e085d51-f41d-43ff-aa62-011430d7d2eb service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:45.996438 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:45.997298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:45.997827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:45.998201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:46.000268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:46.093122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:46.094543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:46.116669 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1381/2755] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:56:45 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:56:49.385815 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:49.387323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:49.402580 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa4ca3-78dd-45b9-8199-edaf7842ff77 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:49.408195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:49.408849 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:49.409127 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:49.409396 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] PATH_INFO: `/v3/users/dea49d304629415895813b6ab7f9c9db` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:49.423776 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dea49d304629415895813b6ab7f9c9db)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:49.425729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:49.477386 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dea49d304629415895813b6ab7f9c9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.484270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dea49d304629415895813b6ab7f9c9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.484935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dea49d304629415895813b6ab7f9c9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.493422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dea49d304629415895813b6ab7f9c9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.506188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dea49d304629415895813b6ab7f9c9db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.512600 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1375/2756] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:56:49 2022] DELETE /identity/v3/users/dea49d304629415895813b6ab7f9c9db => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:56:49.703821 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:49.717096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f5a88-5292-4add-9cd7-43afb07ca875 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:49.735049 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:56:49.774404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:49.775036 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:49.775313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:49.775579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] PATH_INFO: `/v3/projects/93588cc1d2704a73ae5a76dcc6fc44f1` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:49.785724 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93588cc1d2704a73ae5a76dcc6fc44f1)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:49.787653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:49.836757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93588cc1d2704a73ae5a76dcc6fc44f1', 'request_id': 'req-ef98752d-c1eb-46a9-b7e8-51776f268241'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.843858 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93588cc1d2704a73ae5a76dcc6fc44f1', 'request_id': 'req-ef98752d-c1eb-46a9-b7e8-51776f268241'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.853095 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93588cc1d2704a73ae5a76dcc6fc44f1', 'request_id': 'req-ef98752d-c1eb-46a9-b7e8-51776f268241'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.869318 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] The token cache is being invalidate because project 93588cc1d2704a73ae5a76dcc6fc44f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:56:49.869503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.870186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:49.871714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1382/2757] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:56:49 2022] DELETE /identity/v3/projects/93588cc1d2704a73ae5a76dcc6fc44f1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:56:49.875976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:49.877442 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:49.891366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04c9dbc-c12f-40f6-b783-1b42fe9397e1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:49.939647 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:56:49.950590 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:49.951240 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:49.951690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:49.952084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] PATH_INFO: `/v3/users/e7e9578eec534990ba636c779ca93259` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:49.966356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7e9578eec534990ba636c779ca93259)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:49.968290 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:50.021394 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7e9578eec534990ba636c779ca93259'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.029172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7e9578eec534990ba636c779ca93259'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.029908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7e9578eec534990ba636c779ca93259'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.038248 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7e9578eec534990ba636c779ca93259'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.052015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7e9578eec534990ba636c779ca93259'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.058071 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1376/2758] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:56:49 2022] DELETE /identity/v3/users/e7e9578eec534990ba636c779ca93259 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:56:50.397111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:56:50.398679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:56:50.412597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef98752d-c1eb-46a9-b7e8-51776f268241 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:56:50.476284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:56:50.476928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:56:50.477208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:56:50.477474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] PATH_INFO: `/v3/projects/dc21568194bb4facbc9c971671462aa3` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:56:50.488748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc21568194bb4facbc9c971671462aa3)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:56:50.490788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:56:50.533006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:56:50.546365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc21568194bb4facbc9c971671462aa3', 'request_id': 'req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.551795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc21568194bb4facbc9c971671462aa3', 'request_id': 'req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.558722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc21568194bb4facbc9c971671462aa3', 'request_id': 'req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.572710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] The token cache is being invalidate because project dc21568194bb4facbc9c971671462aa3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:56:50.573011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.573699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:56:50.575354 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1383/2759] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:56:50 2022] DELETE /identity/v3/projects/dc21568194bb4facbc9c971671462aa3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:57:00.054929 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:57:00.569811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:58:51.035984 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:58:51.037749 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:51.104124 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e782e3-5855-4b66-9f55-360f65124fbc admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:58:51.154722 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:58:51.155463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:51.155874 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:51.156141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:51.158288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:51.168073 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:58:51.169480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:58:51.177380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1377/2760] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:58:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:58:56.455819 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:58:56.457784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:56.475331 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b5c50-2b64-4cd6-a2a3-79e1785e15ab admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:58:56.480437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086761d8-b02e-4153-8ec4-5a12207d137e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:58:56.481167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086761d8-b02e-4153-8ec4-5a12207d137e service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:56.481445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086761d8-b02e-4153-8ec4-5a12207d137e service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:56.481726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086761d8-b02e-4153-8ec4-5a12207d137e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:56.483827 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-086761d8-b02e-4153-8ec4-5a12207d137e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:56.578230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086761d8-b02e-4153-8ec4-5a12207d137e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:58:56.579726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086761d8-b02e-4153-8ec4-5a12207d137e service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:58:56.602300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1384/2761] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:58:56.610519 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:58:56.619445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afad7969-6786-47b9-a27e-9086f5f05851 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:58:56.624241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706a50-00b9-451c-a4c7-8f68ed458c8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:58:56.624844 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706a50-00b9-451c-a4c7-8f68ed458c8e service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:56.625121 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706a50-00b9-451c-a4c7-8f68ed458c8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:56.625385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc706a50-00b9-451c-a4c7-8f68ed458c8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:56.627707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-dc706a50-00b9-451c-a4c7-8f68ed458c8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-dc706a50-00b9-451c-a4c7-8f68ed458c8e service neutron] Could not find project: 31137b9776dd4dbf89dcc7b10ef4e65b.: keystone.exception.ProjectNotFound: Could not find project: 31137b9776dd4dbf89dcc7b10ef4e65b. Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 05 14:58:56.640872 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:58:56.642072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:58:56.643218 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:58:56.644366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:58:56.645489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 31137b9776dd4dbf89dcc7b10ef4e65b. Oct 05 14:58:56.645489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:58:56.645489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1378/2762] 10.222.0.23 () {60 vars in 1396 bytes} [Wed Oct 5 14:58:56 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 113 bytes in 32 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 05 14:58:56.679672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56cedd-c101-4734-949d-231a268d5ee8 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:56.679949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56cedd-c101-4734-949d-231a268d5ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:56.680215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56cedd-c101-4734-949d-231a268d5ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:56.738415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-4d56cedd-c101-4734-949d-231a268d5ee8 None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:58:56.739529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d56cedd-c101-4734-949d-231a268d5ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:56.751949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1385/2763] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:58:56 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:58:56.766819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706a50-00b9-451c-a4c7-8f68ed458c8e service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:58:56.776698 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc706a50-00b9-451c-a4c7-8f68ed458c8e service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:58:56.781045 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba1df1-2529-4706-85bc-67c01903a471 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:58:56.781736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1df1-2529-4706-85bc-67c01903a471 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:56.781901 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1df1-2529-4706-85bc-67c01903a471 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:56.782198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba1df1-2529-4706-85bc-67c01903a471 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:56.793146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba1df1-2529-4706-85bc-67c01903a471 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:58:56.794601 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba1df1-2529-4706-85bc-67c01903a471 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:58:56.805724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1379/2764] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:58:56.893971 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56cedd-c101-4734-949d-231a268d5ee8 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:58:56.895637 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d56cedd-c101-4734-949d-231a268d5ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:56.910624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56cedd-c101-4734-949d-231a268d5ee8 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:58:56.916698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:58:56.917411 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:56.917715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:56.917976 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:56.920019 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:56.932522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:58:56.934202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:58:56.944839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1386/2765] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:58:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:58:57.266261 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba1df1-2529-4706-85bc-67c01903a471 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:58:57.278405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba1df1-2529-4706-85bc-67c01903a471 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:58:57.286087 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb21da-6403-4077-960c-a9d47b813b43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:58:57.286712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb21da-6403-4077-960c-a9d47b813b43 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:57.286982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb21da-6403-4077-960c-a9d47b813b43 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:57.287256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bb21da-6403-4077-960c-a9d47b813b43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:57.289387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-69bb21da-6403-4077-960c-a9d47b813b43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application [None req-69bb21da-6403-4077-960c-a9d47b813b43 service nova] Could not find project: 5591c836050d488487149a6f548393c9.: keystone.exception.ProjectNotFound: Could not find project: 5591c836050d488487149a6f548393c9. Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 05 14:58:57.301803 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:58:57.303281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:58:57.304654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application with Lock( Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:58:57.305801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:58:57.306996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 5591c836050d488487149a6f548393c9. Oct 05 14:58:57.306996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: ERROR keystone.server.flask.application Oct 05 14:58:57.306996 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1380/2766] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:58:57 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 37 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 05 14:58:57.348698 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:58:57.359909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d679eb86-94de-4b52-b5cf-0b458211d5f7 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:58:57.366040 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457b755-e593-4f7d-839b-9c4cef2a8376 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:58:57.367594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457b755-e593-4f7d-839b-9c4cef2a8376 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:57.367594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457b755-e593-4f7d-839b-9c4cef2a8376 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:57.368093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c457b755-e593-4f7d-839b-9c4cef2a8376 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:57.370449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-c457b755-e593-4f7d-839b-9c4cef2a8376 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application [None req-c457b755-e593-4f7d-839b-9c4cef2a8376 service nova] Could not find project: 93588cc1d2704a73ae5a76dcc6fc44f1.: keystone.exception.ProjectNotFound: Could not find project: 93588cc1d2704a73ae5a76dcc6fc44f1. Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Oct 05 14:58:57.385907 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:58:57.387200 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self._validate_token_resources() Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 05 14:58:57.388726 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.get_or_create( Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application with Lock( Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._enter() Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.creator() Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application created_value = creator( Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 05 14:58:57.390055 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 05 14:58:57.391882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 93588cc1d2704a73ae5a76dcc6fc44f1. Oct 05 14:58:57.391882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: ERROR keystone.server.flask.application Oct 05 14:58:57.391882 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1387/2767] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:58:57 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 39 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Oct 05 14:58:58.132853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb21da-6403-4077-960c-a9d47b813b43 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:58:58.149687 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bb21da-6403-4077-960c-a9d47b813b43 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:58:58.155562 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:58:58.156154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:58:58.156420 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:58:58.156683 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:58:58.158784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:58:58.237665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:58:58.239292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:58:58.254017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:58:58.259986 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1381/2768] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:58:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:02.196186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457b755-e593-4f7d-839b-9c4cef2a8376 service nova] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:02.206335 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c457b755-e593-4f7d-839b-9c4cef2a8376 service nova] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:02.211170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:02.211781 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:02.212048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:02.212317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:02.214397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:02.297773 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:02.311772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:02.313220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:02.333751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1388/2769] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:03.316499 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:03.335769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94ea8a-c87c-4d9c-bf67-3eb5557c0de3 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:03.344444 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:03.345178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:03.345455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:03.345772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:03.347843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:03.362472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:03.364167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:03.376477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1382/2770] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:06.804679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:06.806127 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:06.818790 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3ae589-4cca-4523-83ee-50dd95e9f853 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:06.823743 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:06.824589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:06.824719 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:06.824979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] PATH_INFO: `/v3/users/2e988e0435464c1d8aa4842fab56dfa4` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:06.826198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e988e0435464c1d8aa4842fab56dfa4)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:06.828220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:06.875539 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e988e0435464c1d8aa4842fab56dfa4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:06.882245 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e988e0435464c1d8aa4842fab56dfa4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:06.882884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e988e0435464c1d8aa4842fab56dfa4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:06.890024 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e988e0435464c1d8aa4842fab56dfa4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:06.904826 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d4d0685-4259-4bfa-83ce-1b354a38ee39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e988e0435464c1d8aa4842fab56dfa4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:06.912238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1389/2771] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:59:06 2022] DELETE /identity/v3/users/2e988e0435464c1d8aa4842fab56dfa4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:07.103696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:07.114639 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d7d5d1-1653-4a48-aa9c-b61f917bb1b5 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:07.172268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:07.172915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.173194 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.173466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] PATH_INFO: `/v3/projects/45170bafd3c5447cbe6bbb69d7c5f544` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.174619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45170bafd3c5447cbe6bbb69d7c5f544)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:07.176538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:07.235408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45170bafd3c5447cbe6bbb69d7c5f544', 'request_id': 'req-94b9485e-9549-4b77-b25b-c7c267fac029'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:07.241138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45170bafd3c5447cbe6bbb69d7c5f544', 'request_id': 'req-94b9485e-9549-4b77-b25b-c7c267fac029'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:07.247587 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45170bafd3c5447cbe6bbb69d7c5f544', 'request_id': 'req-94b9485e-9549-4b77-b25b-c7c267fac029'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:07.262402 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] The token cache is being invalidate because project 45170bafd3c5447cbe6bbb69d7c5f544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:59:07.262763 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:07.263404 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-94b9485e-9549-4b77-b25b-c7c267fac029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:07.264921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1383/2772] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:59:07 2022] DELETE /identity/v3/projects/45170bafd3c5447cbe6bbb69d7c5f544 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:07.271001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c5dde-6903-4988-9783-1382d98e1b0d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.271284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c5dde-6903-4988-9783-1382d98e1b0d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.271561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c5dde-6903-4988-9783-1382d98e1b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.311711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-da9c5dde-6903-4988-9783-1382d98e1b0d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:07.312925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-da9c5dde-6903-4988-9783-1382d98e1b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:07.375902 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1390/2773] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:07.385061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bf45b-46de-4986-9a30-e0a0684949bf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.385325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bf45b-46de-4986-9a30-e0a0684949bf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.385659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897bf45b-46de-4986-9a30-e0a0684949bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.406690 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-897bf45b-46de-4986-9a30-e0a0684949bf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:07.407930 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-897bf45b-46de-4986-9a30-e0a0684949bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:07.416809 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1384/2774] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:07.421208 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c5dde-6903-4988-9783-1382d98e1b0d None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:07.431282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c5dde-6903-4988-9783-1382d98e1b0d None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:07.437363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:07.438004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.438259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.438520 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.439146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:59:07.439441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:07.441033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:07.452915 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1391/2775] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:59:07.457326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bf45b-46de-4986-9a30-e0a0684949bf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:07.467088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897bf45b-46de-4986-9a30-e0a0684949bf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:07.472225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb322a06-6602-4262-baaa-57f0eca7c14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:07.472820 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb322a06-6602-4262-baaa-57f0eca7c14c admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.473094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb322a06-6602-4262-baaa-57f0eca7c14c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.473364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb322a06-6602-4262-baaa-57f0eca7c14c admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.474052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb322a06-6602-4262-baaa-57f0eca7c14c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:07.475933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb322a06-6602-4262-baaa-57f0eca7c14c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:07.530704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1385/2776] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:59:07 2022] POST /identity/v3/projects => generated 385 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:07.535194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:07.546355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd57ab-39be-40a2-a3db-e7f867ba240a admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:07.567974 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:07.609029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf5aa2-b2a3-4e67-9cd3-493bed15f881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:07.609654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf5aa2-b2a3-4e67-9cd3-493bed15f881 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.609953 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf5aa2-b2a3-4e67-9cd3-493bed15f881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.610215 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf5aa2-b2a3-4e67-9cd3-493bed15f881 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.610854 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf5aa2-b2a3-4e67-9cd3-493bed15f881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:07.612798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf5aa2-b2a3-4e67-9cd3-493bed15f881 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:07.639838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1392/2777] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:59:07 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:59:07.641268 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb322a06-6602-4262-baaa-57f0eca7c14c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:07.652263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb322a06-6602-4262-baaa-57f0eca7c14c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:07.658466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb67fca-0a81-4e8e-be07-608b01574179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:07.659619 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb67fca-0a81-4e8e-be07-608b01574179 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.659965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb67fca-0a81-4e8e-be07-608b01574179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.660302 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb67fca-0a81-4e8e-be07-608b01574179 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.661043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb67fca-0a81-4e8e-be07-608b01574179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:07.662783 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb67fca-0a81-4e8e-be07-608b01574179 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:07.674493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1386/2778] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:07.678810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf5aa2-b2a3-4e67-9cd3-493bed15f881 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:07.689365 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf5aa2-b2a3-4e67-9cd3-493bed15f881 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:07.695254 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd5864-8894-4d27-8a4c-577025b5a7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:07.695962 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5864-8894-4d27-8a4c-577025b5a7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.696267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5864-8894-4d27-8a4c-577025b5a7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.696560 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd5864-8894-4d27-8a4c-577025b5a7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.697286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd5864-8894-4d27-8a4c-577025b5a7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:07.698936 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcd5864-8894-4d27-8a4c-577025b5a7d9 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:07.710169 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1393/2779] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:59:07.714655 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb67fca-0a81-4e8e-be07-608b01574179 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:07.725388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb67fca-0a81-4e8e-be07-608b01574179 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:07.730406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:07.731065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.731403 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.731675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] PATH_INFO: `/v3/projects/80d855ec2539489a9565cc2892315976/users/d1f3c7f91835440fafbf607c947b9139/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.748513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d855ec2539489a9565cc2892315976, user_id=d1f3c7f91835440fafbf607c947b9139, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:07.751414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:07.767915 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1387/2780] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:07 2022] PUT /identity/v3/projects/80d855ec2539489a9565cc2892315976/users/d1f3c7f91835440fafbf607c947b9139/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:59:07.778803 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c81a2e-9a42-44a8-b644-903b60e02f33 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.779472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c81a2e-9a42-44a8-b644-903b60e02f33 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.779711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c81a2e-9a42-44a8-b644-903b60e02f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:07.814548 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-85c81a2e-9a42-44a8-b644-903b60e02f33 None None] MFA Rules not processed for user `d1f3c7f91835440fafbf607c947b9139`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:07.815786 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-85c81a2e-9a42-44a8-b644-903b60e02f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:07.894577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1394/2781] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:07.905349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:07.907178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:07.913272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:07.990340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60856c35-40c2-490f-b4d0-cddf25101b09 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:07.996617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:07.997310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:07.997573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:07.997891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:08.006306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:08.007914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:08.017621 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1388/2782] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:09.822879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782d7f2-42ba-4c27-83a4-5c60eb2eb195 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:09.823366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782d7f2-42ba-4c27-83a4-5c60eb2eb195 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:09.823366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8782d7f2-42ba-4c27-83a4-5c60eb2eb195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:09.856380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8782d7f2-42ba-4c27-83a4-5c60eb2eb195 None None] MFA Rules not processed for user `7de0d01f891d4d73bc44810aacbca477`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:09.857702 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8782d7f2-42ba-4c27-83a4-5c60eb2eb195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:09.927429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1395/2783] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:10.247148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:10.248746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:10.311123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3f9eb-cee6-4951-a8cc-9bcd2c84c2f0 service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:10.315667 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cf46d7-4414-4727-9f61-1e124c0a71c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:10.316333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf46d7-4414-4727-9f61-1e124c0a71c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:10.316608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf46d7-4414-4727-9f61-1e124c0a71c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:10.316871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cf46d7-4414-4727-9f61-1e124c0a71c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:10.318851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-19cf46d7-4414-4727-9f61-1e124c0a71c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:10.329479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf46d7-4414-4727-9f61-1e124c0a71c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:10.330962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cf46d7-4414-4727-9f61-1e124c0a71c4 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:10.339792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1389/2784] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:10.683139 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782d7f2-42ba-4c27-83a4-5c60eb2eb195 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:10.684832 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8782d7f2-42ba-4c27-83a4-5c60eb2eb195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:10.725360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:10.762057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8782d7f2-42ba-4c27-83a4-5c60eb2eb195 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:10.767600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:10.768246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:10.768561 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:10.768906 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:10.770870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:10.783156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:10.784789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:10.794965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1396/2785] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:20.335804 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=59 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:20.790917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=18 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:23.301477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3bf4-9b8b-4397-8210-bf5ee1617a74 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:23.302250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3bf4-9b8b-4397-8210-bf5ee1617a74 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:23.302250 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de3bf4-9b8b-4397-8210-bf5ee1617a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:23.331263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-89de3bf4-9b8b-4397-8210-bf5ee1617a74 None None] MFA Rules not processed for user `bc27904d964e43438c32876b8a1b202d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:23.332473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89de3bf4-9b8b-4397-8210-bf5ee1617a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:23.399846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1390/2786] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:24.545295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:24.625251 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806a4ba-cad6-43b9-bbd6-963da9083f19 service placement] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:24.631692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:24.632340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:24.632643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:24.632959 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] PATH_INFO: `/v3/users/d1f3c7f91835440fafbf607c947b9139` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:24.634292 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1f3c7f91835440fafbf607c947b9139)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:24.636230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:24.689930 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1f3c7f91835440fafbf607c947b9139'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:24.712654 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1f3c7f91835440fafbf607c947b9139'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:24.713396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1f3c7f91835440fafbf607c947b9139'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:24.723931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1f3c7f91835440fafbf607c947b9139'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:24.748389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1f3c7f91835440fafbf607c947b9139'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:24.758207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1397/2787] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:59:24 2022] DELETE /identity/v3/users/d1f3c7f91835440fafbf607c947b9139 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:24.761987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a504073-fe44-4c30-91fb-786991becc89 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:24.762689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a504073-fe44-4c30-91fb-786991becc89 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:24.762689 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a504073-fe44-4c30-91fb-786991becc89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:24.782155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-2a504073-fe44-4c30-91fb-786991becc89 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:24.783384 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2a504073-fe44-4c30-91fb-786991becc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:24.846968 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1391/2788] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:59:24.856921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:24.858478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:24.918894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4873287-a28a-4f6f-811b-39f7f386c0e2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:24.922688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9537c-738c-41a7-afb0-c6d19044b536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:24.923304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9537c-738c-41a7-afb0-c6d19044b536 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:24.923577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9537c-738c-41a7-afb0-c6d19044b536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:24.923844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9537c-738c-41a7-afb0-c6d19044b536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:24.933284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9537c-738c-41a7-afb0-c6d19044b536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:24.934757 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9537c-738c-41a7-afb0-c6d19044b536 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:24.944451 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1398/2789] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:25.226537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a504073-fe44-4c30-91fb-786991becc89 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:25.228064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-2a504073-fe44-4c30-91fb-786991becc89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:25.241170 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a504073-fe44-4c30-91fb-786991becc89 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:25.246718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:25.247362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.247784 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.248053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] PATH_INFO: `/v3/projects/80d855ec2539489a9565cc2892315976` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.249195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d855ec2539489a9565cc2892315976)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:25.251128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:25.301918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d855ec2539489a9565cc2892315976', 'request_id': 'req-36fa38b6-bdfe-456b-8c80-21ffae6d3616'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:25.307977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d855ec2539489a9565cc2892315976', 'request_id': 'req-36fa38b6-bdfe-456b-8c80-21ffae6d3616'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:25.314661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d855ec2539489a9565cc2892315976', 'request_id': 'req-36fa38b6-bdfe-456b-8c80-21ffae6d3616'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:25.329034 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] The token cache is being invalidate because project 80d855ec2539489a9565cc2892315976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:59:25.329456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:25.330206 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-36fa38b6-bdfe-456b-8c80-21ffae6d3616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:25.332255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1392/2790] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:59:25 2022] DELETE /identity/v3/projects/80d855ec2539489a9565cc2892315976 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:25.337878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bbaba-87bf-4996-b2a6-eea6ef55f37e None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.338167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bbaba-87bf-4996-b2a6-eea6ef55f37e None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.338527 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bbaba-87bf-4996-b2a6-eea6ef55f37e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.370843 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f71bbaba-87bf-4996-b2a6-eea6ef55f37e None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:25.371440 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f71bbaba-87bf-4996-b2a6-eea6ef55f37e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:25.392823 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:25.447052 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1399/2791] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:59:25.455037 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d2eca-185b-420c-999a-5d1d6e0ceedf None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.455388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d2eca-185b-420c-999a-5d1d6e0ceedf None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.455727 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634d2eca-185b-420c-999a-5d1d6e0ceedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.476938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-634d2eca-185b-420c-999a-5d1d6e0ceedf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:25.478198 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-634d2eca-185b-420c-999a-5d1d6e0ceedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:25.487329 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1393/2792] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:25.491705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bbaba-87bf-4996-b2a6-eea6ef55f37e None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:25.502353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bbaba-87bf-4996-b2a6-eea6ef55f37e None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:25.507186 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:25.507820 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.508094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.508362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.509017 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:59:25.509316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:25.510979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:25.526175 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1400/2793] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:59:25.530706 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d2eca-185b-420c-999a-5d1d6e0ceedf None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:25.540471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634d2eca-185b-420c-999a-5d1d6e0ceedf None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:25.545952 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202f638-ad3a-4972-b9d5-34d2300d93d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:25.546735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202f638-ad3a-4972-b9d5-34d2300d93d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.547428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202f638-ad3a-4972-b9d5-34d2300d93d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.547428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f202f638-ad3a-4972-b9d5-34d2300d93d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.548114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202f638-ad3a-4972-b9d5-34d2300d93d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:25.550440 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f202f638-ad3a-4972-b9d5-34d2300d93d0 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:25.586715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:25.610565 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1394/2794] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:59:25 2022] POST /identity/v3/projects => generated 357 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:25.611460 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:25.622867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0345b-cc2c-44ab-a0d1-f54c6ef5cd93 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:25.697514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00375dce-73e0-4339-80a8-3ee3428a191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:25.698156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00375dce-73e0-4339-80a8-3ee3428a191b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.698427 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00375dce-73e0-4339-80a8-3ee3428a191b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.698694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00375dce-73e0-4339-80a8-3ee3428a191b admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.699323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00375dce-73e0-4339-80a8-3ee3428a191b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:25.701285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00375dce-73e0-4339-80a8-3ee3428a191b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:25.724796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1401/2795] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:59:25 2022] POST /identity/v3/users => generated 279 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:59:25.728554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202f638-ad3a-4972-b9d5-34d2300d93d0 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:25.739480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f202f638-ad3a-4972-b9d5-34d2300d93d0 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:25.744358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f51d4b-ebe0-4323-87fe-aa08a2241588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:25.744992 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f51d4b-ebe0-4323-87fe-aa08a2241588 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.745263 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f51d4b-ebe0-4323-87fe-aa08a2241588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.745549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f51d4b-ebe0-4323-87fe-aa08a2241588 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.746236 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f51d4b-ebe0-4323-87fe-aa08a2241588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:25.748150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f51d4b-ebe0-4323-87fe-aa08a2241588 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:25.759792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1395/2796] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:25.764117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00375dce-73e0-4339-80a8-3ee3428a191b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:25.774866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00375dce-73e0-4339-80a8-3ee3428a191b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:25.780378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9141d-6e70-4d44-9afd-1f5e425c182e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:25.781026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9141d-6e70-4d44-9afd-1f5e425c182e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.781301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9141d-6e70-4d44-9afd-1f5e425c182e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.781666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9141d-6e70-4d44-9afd-1f5e425c182e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.782313 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9141d-6e70-4d44-9afd-1f5e425c182e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:25.783951 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9141d-6e70-4d44-9afd-1f5e425c182e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:25.796256 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1402/2797] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:25.800599 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f51d4b-ebe0-4323-87fe-aa08a2241588 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:25.810610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f51d4b-ebe0-4323-87fe-aa08a2241588 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:25.815646 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5652f4b-7497-4980-a2a2-bcda09af6703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:25.816306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5652f4b-7497-4980-a2a2-bcda09af6703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.816569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5652f4b-7497-4980-a2a2-bcda09af6703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.816834 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5652f4b-7497-4980-a2a2-bcda09af6703 admin admin] PATH_INFO: `/v3/projects/b7a477e1eb3f4d209275ec43c801c3bf/users/ef70647f6b2c4dd39a133ca16c6c3441/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.832077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5652f4b-7497-4980-a2a2-bcda09af6703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7a477e1eb3f4d209275ec43c801c3bf, user_id=ef70647f6b2c4dd39a133ca16c6c3441, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:25.834806 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5652f4b-7497-4980-a2a2-bcda09af6703 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:25.849395 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1396/2798] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:25 2022] PUT /identity/v3/projects/b7a477e1eb3f4d209275ec43c801c3bf/users/ef70647f6b2c4dd39a133ca16c6c3441/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:59:25.860787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9091-a44d-42a8-af3f-4e62bfb40b81 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:25.861196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9091-a44d-42a8-af3f-4e62bfb40b81 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:25.861472 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac9091-a44d-42a8-af3f-4e62bfb40b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:25.889584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-f3ac9091-a44d-42a8-af3f-4e62bfb40b81 None None] MFA Rules not processed for user `ef70647f6b2c4dd39a133ca16c6c3441`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:25.890921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-f3ac9091-a44d-42a8-af3f-4e62bfb40b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:25.958546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1403/2799] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:25 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:25.963272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5652f4b-7497-4980-a2a2-bcda09af6703 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.028582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5652f4b-7497-4980-a2a2-bcda09af6703 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.033400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7666c5-2da0-4267-aad4-182a205b8a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.034172 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7666c5-2da0-4267-aad4-182a205b8a3b admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.034445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7666c5-2da0-4267-aad4-182a205b8a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.034707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7666c5-2da0-4267-aad4-182a205b8a3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.035354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7666c5-2da0-4267-aad4-182a205b8a3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.037281 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7666c5-2da0-4267-aad4-182a205b8a3b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.066886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1397/2800] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:59:25 2022] POST /identity/v3/projects => generated 357 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:26.071551 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac9091-a44d-42a8-af3f-4e62bfb40b81 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.082542 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac9091-a44d-42a8-af3f-4e62bfb40b81 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.096603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:26.165920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543e158-f775-48f7-910b-a90d552dff63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.166986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e158-f775-48f7-910b-a90d552dff63 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.167470 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e158-f775-48f7-910b-a90d552dff63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.167949 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e158-f775-48f7-910b-a90d552dff63 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.169115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543e158-f775-48f7-910b-a90d552dff63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.172825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543e158-f775-48f7-910b-a90d552dff63 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.206317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1404/2801] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:59:26 2022] POST /identity/v3/users => generated 285 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:26.213014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7666c5-2da0-4267-aad4-182a205b8a3b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.224735 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7666c5-2da0-4267-aad4-182a205b8a3b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.230994 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbf36d-1bf7-4682-941d-41155dc234b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.231614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbf36d-1bf7-4682-941d-41155dc234b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.231886 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbf36d-1bf7-4682-941d-41155dc234b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.232154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbf36d-1bf7-4682-941d-41155dc234b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.232798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbf36d-1bf7-4682-941d-41155dc234b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.234437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbf36d-1bf7-4682-941d-41155dc234b1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.246192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1398/2802] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:26.250876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543e158-f775-48f7-910b-a90d552dff63 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.261653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543e158-f775-48f7-910b-a90d552dff63 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.266893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75016d52-6e66-473b-9566-d041cf67d8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.267779 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75016d52-6e66-473b-9566-d041cf67d8c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.268079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75016d52-6e66-473b-9566-d041cf67d8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.268371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75016d52-6e66-473b-9566-d041cf67d8c8 admin admin] PATH_INFO: `/v3/projects/a2792d52619148b0b9d1cae2ae6bd901/users/1b695d6141b44fc0b1719be0ffe3971c/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.284507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75016d52-6e66-473b-9566-d041cf67d8c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2792d52619148b0b9d1cae2ae6bd901, user_id=1b695d6141b44fc0b1719be0ffe3971c, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.287158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75016d52-6e66-473b-9566-d041cf67d8c8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.300835 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1405/2803] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:26 2022] PUT /identity/v3/projects/a2792d52619148b0b9d1cae2ae6bd901/users/1b695d6141b44fc0b1719be0ffe3971c/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:26.306256 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbf36d-1bf7-4682-941d-41155dc234b1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.316538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbf36d-1bf7-4682-941d-41155dc234b1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.374222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dee84a-6ffc-4719-9247-39fb46a9474c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.374961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dee84a-6ffc-4719-9247-39fb46a9474c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.375323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dee84a-6ffc-4719-9247-39fb46a9474c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.375594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dee84a-6ffc-4719-9247-39fb46a9474c admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.376294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dee84a-6ffc-4719-9247-39fb46a9474c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.377958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dee84a-6ffc-4719-9247-39fb46a9474c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.388195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1399/2804] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:26 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:59:26.392788 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75016d52-6e66-473b-9566-d041cf67d8c8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.404626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75016d52-6e66-473b-9566-d041cf67d8c8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.410488 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.411155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.411431 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.411693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] PATH_INFO: `/v3/projects/a2792d52619148b0b9d1cae2ae6bd901/users/1b695d6141b44fc0b1719be0ffe3971c/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.413957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2792d52619148b0b9d1cae2ae6bd901, user_id=1b695d6141b44fc0b1719be0ffe3971c, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.416730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.427655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1406/2805] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:26 2022] PUT /identity/v3/projects/a2792d52619148b0b9d1cae2ae6bd901/users/1b695d6141b44fc0b1719be0ffe3971c/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:59:26.438374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d35a4b-7225-47ca-8c66-cb2cc00c808d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.438614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d35a4b-7225-47ca-8c66-cb2cc00c808d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.438877 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d35a4b-7225-47ca-8c66-cb2cc00c808d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.469688 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-e1d35a4b-7225-47ca-8c66-cb2cc00c808d None None] MFA Rules not processed for user `1b695d6141b44fc0b1719be0ffe3971c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:26.470989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-e1d35a4b-7225-47ca-8c66-cb2cc00c808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:26.494808 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:26.561246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1400/2806] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:26 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:26.575818 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.577730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:26.661115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8528a-d151-49ec-b127-b53e169cf4b4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.666865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.667894 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.668044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.668426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.681445 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.682886 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.695405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1407/2807] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:26 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:26.780495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d35a4b-7225-47ca-8c66-cb2cc00c808d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.790245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d35a4b-7225-47ca-8c66-cb2cc00c808d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.793904 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.794507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.794780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.795046 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.822309 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:26.826666 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:26.873935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.875989 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.886347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1401/2808] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:26 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:59:26.917851 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836ae793-9d86-4ddc-be25-15ae2769d268 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:26.924808 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190f3982-220b-4f6c-ab5d-cfffa4c0e718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:26.925503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f3982-220b-4f6c-ab5d-cfffa4c0e718 service nova] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:26.925888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f3982-220b-4f6c-ab5d-cfffa4c0e718 service nova] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:26.926239 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190f3982-220b-4f6c-ab5d-cfffa4c0e718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:26.928255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-190f3982-220b-4f6c-ab5d-cfffa4c0e718 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:26.943172 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f3982-220b-4f6c-ab5d-cfffa4c0e718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:26.945395 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190f3982-220b-4f6c-ab5d-cfffa4c0e718 service nova] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:26.951647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:26.962506 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1408/2809] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:59:36.881401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=49 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:36.963206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:40.440107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:40.441668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:40.501385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f88dd8-a96c-4ed0-a9cd-ad5da780445e service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:40.509940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:40.510515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:40.510913 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:40.511213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:40.513114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:40.527405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:40.528812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:40.540015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1402/2810] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:40.552799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b641d-be59-4057-a8c9-b9213143fda4 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:40.552799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b641d-be59-4057-a8c9-b9213143fda4 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:40.552799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b641d-be59-4057-a8c9-b9213143fda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:40.574288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a88b641d-be59-4057-a8c9-b9213143fda4 None None] MFA Rules not processed for user `83f4897f360f4f23aac5749685fdf500`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:40.575512 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a88b641d-be59-4057-a8c9-b9213143fda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:40.585570 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1409/2811] 10.222.0.23 () {58 vars in 1009 bytes} [Wed Oct 5 14:59:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:40.596418 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:40.606798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa306862-eb9f-4352-861f-89a7eb9114e8 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:40.610788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ee1a0a-323f-4d44-a362-ca487d5d4c5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:40.611436 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee1a0a-323f-4d44-a362-ca487d5d4c5f service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:40.611712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee1a0a-323f-4d44-a362-ca487d5d4c5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:40.611981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ee1a0a-323f-4d44-a362-ca487d5d4c5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:40.623561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee1a0a-323f-4d44-a362-ca487d5d4c5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:40.624995 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ee1a0a-323f-4d44-a362-ca487d5d4c5f service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:40.636207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1403/2812] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:42.101493 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b641d-be59-4057-a8c9-b9213143fda4 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:42.165794 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b641d-be59-4057-a8c9-b9213143fda4 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:42.171180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:42.171813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:42.172093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:42.172374 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] PATH_INFO: `/v3/users/ef70647f6b2c4dd39a133ca16c6c3441` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:42.173611 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef70647f6b2c4dd39a133ca16c6c3441)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:42.175584 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:42.224816 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef70647f6b2c4dd39a133ca16c6c3441'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.231806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef70647f6b2c4dd39a133ca16c6c3441'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.232482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef70647f6b2c4dd39a133ca16c6c3441'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.240219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef70647f6b2c4dd39a133ca16c6c3441'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.255667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef70647f6b2c4dd39a133ca16c6c3441'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.262010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1410/2813] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:59:42 2022] DELETE /identity/v3/users/ef70647f6b2c4dd39a133ca16c6c3441 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:42.266665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27831633-77ac-4cab-a348-2b0f9c176eb1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:42.267161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27831633-77ac-4cab-a348-2b0f9c176eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:42.267161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27831633-77ac-4cab-a348-2b0f9c176eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:42.287049 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-27831633-77ac-4cab-a348-2b0f9c176eb1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:42.288243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27831633-77ac-4cab-a348-2b0f9c176eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:42.350447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1404/2814] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:42.360380 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:42.361957 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:42.426450 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d16ac7d-06fd-4eb4-a845-63c0ae8bbfd0 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:42.433010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf3567-5c4a-4fde-a461-83bc65bb69c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:42.433940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf3567-5c4a-4fde-a461-83bc65bb69c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:42.434230 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf3567-5c4a-4fde-a461-83bc65bb69c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:42.434530 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf3567-5c4a-4fde-a461-83bc65bb69c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:42.449177 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf3567-5c4a-4fde-a461-83bc65bb69c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:42.450668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf3567-5c4a-4fde-a461-83bc65bb69c8 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:42.460170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1411/2815] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:42.732861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27831633-77ac-4cab-a348-2b0f9c176eb1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:42.734387 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-27831633-77ac-4cab-a348-2b0f9c176eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:42.746961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27831633-77ac-4cab-a348-2b0f9c176eb1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:42.752006 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:42.752659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:42.752932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:42.753181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] PATH_INFO: `/v3/projects/b7a477e1eb3f4d209275ec43c801c3bf` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:42.754369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7a477e1eb3f4d209275ec43c801c3bf)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:42.756285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:42.804933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7a477e1eb3f4d209275ec43c801c3bf', 'request_id': 'req-243ab929-24e8-456f-96d7-43d761aee2c2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.810412 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7a477e1eb3f4d209275ec43c801c3bf', 'request_id': 'req-243ab929-24e8-456f-96d7-43d761aee2c2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.816625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7a477e1eb3f4d209275ec43c801c3bf', 'request_id': 'req-243ab929-24e8-456f-96d7-43d761aee2c2'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.830719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] The token cache is being invalidate because project b7a477e1eb3f4d209275ec43c801c3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:59:42.831000 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.831597 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.833064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1405/2816] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:59:42 2022] DELETE /identity/v3/projects/b7a477e1eb3f4d209275ec43c801c3bf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:42.837578 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf3567-5c4a-4fde-a461-83bc65bb69c8 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:42.847478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf3567-5c4a-4fde-a461-83bc65bb69c8 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:42.859978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:42.907694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:42.908326 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:42.908602 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:42.908872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] PATH_INFO: `/v3/users/1b695d6141b44fc0b1719be0ffe3971c` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:42.910031 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b695d6141b44fc0b1719be0ffe3971c)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:42.911950 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:42.962497 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b695d6141b44fc0b1719be0ffe3971c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.975564 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b695d6141b44fc0b1719be0ffe3971c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.976349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b695d6141b44fc0b1719be0ffe3971c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.986123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b695d6141b44fc0b1719be0ffe3971c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:42.999033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-b5688ac1-273f-4abf-ac3c-5d13bce841d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b695d6141b44fc0b1719be0ffe3971c'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:43.005131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1412/2817] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:59:42 2022] DELETE /identity/v3/users/1b695d6141b44fc0b1719be0ffe3971c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:43.281993 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:43.284079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:43.307408 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243ab929-24e8-456f-96d7-43d761aee2c2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:43.318468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:43.369280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:43.369852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.370130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.370443 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] PATH_INFO: `/v3/projects/a2792d52619148b0b9d1cae2ae6bd901` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.371615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2792d52619148b0b9d1cae2ae6bd901)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:43.373524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:43.421474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2792d52619148b0b9d1cae2ae6bd901', 'request_id': 'req-b4f4f830-e434-4a4a-966b-505b3dbcd201'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:43.427410 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2792d52619148b0b9d1cae2ae6bd901', 'request_id': 'req-b4f4f830-e434-4a4a-966b-505b3dbcd201'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:43.434100 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2792d52619148b0b9d1cae2ae6bd901', 'request_id': 'req-b4f4f830-e434-4a4a-966b-505b3dbcd201'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:43.448596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] The token cache is being invalidate because project a2792d52619148b0b9d1cae2ae6bd901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:59:43.448887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:43.449535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-b4f4f830-e434-4a4a-966b-505b3dbcd201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:43.451123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1406/2818] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:59:43 2022] DELETE /identity/v3/projects/a2792d52619148b0b9d1cae2ae6bd901 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:43.456201 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0f45-9d59-4829-9cd2-ccdaafc1e41f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.456699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0f45-9d59-4829-9cd2-ccdaafc1e41f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.456830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f0f45-9d59-4829-9cd2-ccdaafc1e41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.477884 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2c3f0f45-9d59-4829-9cd2-ccdaafc1e41f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:43.479144 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c3f0f45-9d59-4829-9cd2-ccdaafc1e41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:43.543984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1413/2819] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:43.554101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3826fa-63e8-4114-a1cc-bc47817c718c None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.554427 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3826fa-63e8-4114-a1cc-bc47817c718c None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.554747 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3826fa-63e8-4114-a1cc-bc47817c718c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.587753 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1b3826fa-63e8-4114-a1cc-bc47817c718c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:43.589517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1b3826fa-63e8-4114-a1cc-bc47817c718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:43.599368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1407/2820] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:43.603900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f0f45-9d59-4829-9cd2-ccdaafc1e41f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:43.614333 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f0f45-9d59-4829-9cd2-ccdaafc1e41f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:43.620233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:43.620960 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.621320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.621581 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.622271 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:59:43.622565 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:43.624165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:43.637370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1414/2821] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:59:43.643721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3826fa-63e8-4114-a1cc-bc47817c718c None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:43.656459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3826fa-63e8-4114-a1cc-bc47817c718c None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:43.662225 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fbb06-e722-42cb-af08-34587979d342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:43.662824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbb06-e722-42cb-af08-34587979d342 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.663097 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbb06-e722-42cb-af08-34587979d342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.663362 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980fbb06-e722-42cb-af08-34587979d342 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.663998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fbb06-e722-42cb-af08-34587979d342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:43.665945 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980fbb06-e722-42cb-af08-34587979d342 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:43.696773 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1408/2822] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:59:43 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:43.701074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:43.711414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c353a27-afc9-429d-a4e7-cd6bf71fa499 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:43.730658 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:43.769991 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88175b-6218-43cd-8278-0f22cefcaa81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:43.770655 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88175b-6218-43cd-8278-0f22cefcaa81 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.770928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88175b-6218-43cd-8278-0f22cefcaa81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.771197 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c88175b-6218-43cd-8278-0f22cefcaa81 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.771825 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88175b-6218-43cd-8278-0f22cefcaa81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:43.773845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c88175b-6218-43cd-8278-0f22cefcaa81 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:43.796252 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1415/2823] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:59:43 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:43.800585 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fbb06-e722-42cb-af08-34587979d342 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:43.811239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980fbb06-e722-42cb-af08-34587979d342 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:43.816323 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52153d28-129d-4f50-acbe-6f8cd9089897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:43.816931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153d28-129d-4f50-acbe-6f8cd9089897 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.817202 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153d28-129d-4f50-acbe-6f8cd9089897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.817466 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52153d28-129d-4f50-acbe-6f8cd9089897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.818140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52153d28-129d-4f50-acbe-6f8cd9089897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:43.820267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52153d28-129d-4f50-acbe-6f8cd9089897 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:43.831070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1409/2824] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:43.835767 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88175b-6218-43cd-8278-0f22cefcaa81 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:43.846303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c88175b-6218-43cd-8278-0f22cefcaa81 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:43.851480 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559ff5ba-1245-4513-b76b-e1778da358f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:43.852160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff5ba-1245-4513-b76b-e1778da358f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.852435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff5ba-1245-4513-b76b-e1778da358f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.852693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ff5ba-1245-4513-b76b-e1778da358f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.853356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ff5ba-1245-4513-b76b-e1778da358f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:43.855059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559ff5ba-1245-4513-b76b-e1778da358f0 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:43.865955 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1416/2825] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:43.870214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52153d28-129d-4f50-acbe-6f8cd9089897 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:43.880518 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52153d28-129d-4f50-acbe-6f8cd9089897 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:43.885453 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68305-feff-4a74-8a30-ed6ce604cd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:43.886197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68305-feff-4a74-8a30-ed6ce604cd21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.886470 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68305-feff-4a74-8a30-ed6ce604cd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.886759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad68305-feff-4a74-8a30-ed6ce604cd21 admin admin] PATH_INFO: `/v3/projects/cfacb84cbd974aeaa8b9c3d8705e9a0f/users/65e1245b7c3d48108d6a81ef85b8bc81/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.901517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68305-feff-4a74-8a30-ed6ce604cd21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfacb84cbd974aeaa8b9c3d8705e9a0f, user_id=65e1245b7c3d48108d6a81ef85b8bc81, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:43.904148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad68305-feff-4a74-8a30-ed6ce604cd21 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:43.919082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1410/2826] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:43 2022] PUT /identity/v3/projects/cfacb84cbd974aeaa8b9c3d8705e9a0f/users/65e1245b7c3d48108d6a81ef85b8bc81/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:43.929874 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daeb96b-b801-4e69-8923-886ff430f8eb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:43.930165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daeb96b-b801-4e69-8923-886ff430f8eb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:43.930433 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2daeb96b-b801-4e69-8923-886ff430f8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:43.960048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2daeb96b-b801-4e69-8923-886ff430f8eb None None] MFA Rules not processed for user `65e1245b7c3d48108d6a81ef85b8bc81`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:43.961677 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2daeb96b-b801-4e69-8923-886ff430f8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:44.032037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1417/2827] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:44.036762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68305-feff-4a74-8a30-ed6ce604cd21 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.078269 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:44.100534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad68305-feff-4a74-8a30-ed6ce604cd21 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:44.106192 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8383-ddd2-40d5-a7d1-8d895107b50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:44.106829 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8383-ddd2-40d5-a7d1-8d895107b50f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.107130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8383-ddd2-40d5-a7d1-8d895107b50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.107397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5b8383-ddd2-40d5-a7d1-8d895107b50f admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.108030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b8383-ddd2-40d5-a7d1-8d895107b50f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:44.110015 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5b8383-ddd2-40d5-a7d1-8d895107b50f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:44.140768 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1411/2828] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:59:44 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:44.145006 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daeb96b-b801-4e69-8923-886ff430f8eb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.156518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2daeb96b-b801-4e69-8923-886ff430f8eb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:44.213664 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c158-a144-4447-9e76-a9eecc077af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:44.214284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c158-a144-4447-9e76-a9eecc077af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.214543 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c158-a144-4447-9e76-a9eecc077af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.214853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c158-a144-4447-9e76-a9eecc077af2 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.215482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c158-a144-4447-9e76-a9eecc077af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:44.217425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c158-a144-4447-9e76-a9eecc077af2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:44.242963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1418/2829] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:59:44 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:44.248303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8383-ddd2-40d5-a7d1-8d895107b50f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.259657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5b8383-ddd2-40d5-a7d1-8d895107b50f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:44.264858 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c4962-cf2f-4f7b-b9fb-ce62114ac12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:44.265502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c4962-cf2f-4f7b-b9fb-ce62114ac12e admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.265789 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c4962-cf2f-4f7b-b9fb-ce62114ac12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.266056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c4962-cf2f-4f7b-b9fb-ce62114ac12e admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.266712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c4962-cf2f-4f7b-b9fb-ce62114ac12e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:44.268320 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37c4962-cf2f-4f7b-b9fb-ce62114ac12e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:44.279477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1412/2830] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:44.283740 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c158-a144-4447-9e76-a9eecc077af2 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.294294 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c158-a144-4447-9e76-a9eecc077af2 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:44.299414 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b71076-d5a1-4ebc-8dea-d6adf5c03c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:44.300077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b71076-d5a1-4ebc-8dea-d6adf5c03c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.300355 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b71076-d5a1-4ebc-8dea-d6adf5c03c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.300624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b71076-d5a1-4ebc-8dea-d6adf5c03c5b admin admin] PATH_INFO: `/v3/projects/5f14a8ca39224b8b86ec66d54954b08e/users/6c362316ac364a118e063b724a78f70d/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.315922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b71076-d5a1-4ebc-8dea-d6adf5c03c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f14a8ca39224b8b86ec66d54954b08e, user_id=6c362316ac364a118e063b724a78f70d, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:44.318634 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b71076-d5a1-4ebc-8dea-d6adf5c03c5b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:44.321744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:44.336004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1419/2831] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:44 2022] PUT /identity/v3/projects/5f14a8ca39224b8b86ec66d54954b08e/users/6c362316ac364a118e063b724a78f70d/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:44.340118 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c4962-cf2f-4f7b-b9fb-ce62114ac12e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.350349 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c4962-cf2f-4f7b-b9fb-ce62114ac12e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:44.415356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563892e4-3e71-48b3-a1d4-f27a4a9658e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:44.415976 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563892e4-3e71-48b3-a1d4-f27a4a9658e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.416245 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563892e4-3e71-48b3-a1d4-f27a4a9658e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.416513 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563892e4-3e71-48b3-a1d4-f27a4a9658e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.417145 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563892e4-3e71-48b3-a1d4-f27a4a9658e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:44.418793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563892e4-3e71-48b3-a1d4-f27a4a9658e9 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:44.428982 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1413/2832] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:44 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:44.433544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b71076-d5a1-4ebc-8dea-d6adf5c03c5b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.444384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b71076-d5a1-4ebc-8dea-d6adf5c03c5b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:44.449295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:44.450059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.450360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.450627 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] PATH_INFO: `/v3/projects/5f14a8ca39224b8b86ec66d54954b08e/users/6c362316ac364a118e063b724a78f70d/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.452624 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f14a8ca39224b8b86ec66d54954b08e, user_id=6c362316ac364a118e063b724a78f70d, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:44.455305 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:44.466389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1420/2833] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:44 2022] PUT /identity/v3/projects/5f14a8ca39224b8b86ec66d54954b08e/users/6c362316ac364a118e063b724a78f70d/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:44.476509 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8167d3-ffa7-4643-b778-f91e95435724 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.476785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8167d3-ffa7-4643-b778-f91e95435724 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.477052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8167d3-ffa7-4643-b778-f91e95435724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.508244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1b8167d3-ffa7-4643-b778-f91e95435724 None None] MFA Rules not processed for user `6c362316ac364a118e063b724a78f70d`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:44.509447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1b8167d3-ffa7-4643-b778-f91e95435724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:44.578213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1414/2834] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:44.590375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.592357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:44.654987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fdc73f-5f84-4c7e-8937-5c2acfdb009f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:44.661694 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ea957-fbe1-4c65-b11b-e377957ac59b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:44.663207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea957-fbe1-4c65-b11b-e377957ac59b service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.663487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea957-fbe1-4c65-b11b-e377957ac59b service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.663753 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea957-fbe1-4c65-b11b-e377957ac59b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.675401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ea957-fbe1-4c65-b11b-e377957ac59b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:44.677390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ea957-fbe1-4c65-b11b-e377957ac59b service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:44.688703 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1421/2835] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:44 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 14:59:44.720955 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8167d3-ffa7-4643-b778-f91e95435724 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.764407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:44.788981 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8167d3-ffa7-4643-b778-f91e95435724 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:44.794296 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:44.795033 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.795309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.795572 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] PATH_INFO: `/v3/users/65e1245b7c3d48108d6a81ef85b8bc81` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.796817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e1245b7c3d48108d6a81ef85b8bc81)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:44.799367 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:44.849448 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e1245b7c3d48108d6a81ef85b8bc81'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:44.856264 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e1245b7c3d48108d6a81ef85b8bc81'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:44.856956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e1245b7c3d48108d6a81ef85b8bc81'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:44.864943 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e1245b7c3d48108d6a81ef85b8bc81'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:44.881498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e1245b7c3d48108d6a81ef85b8bc81'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:44.887830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1415/2836] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:59:44 2022] DELETE /identity/v3/users/65e1245b7c3d48108d6a81ef85b8bc81 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:44.892510 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b258492-c235-4b52-a70a-406f28081fb9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:44.892766 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b258492-c235-4b52-a70a-406f28081fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:44.893086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b258492-c235-4b52-a70a-406f28081fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:44.912599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9b258492-c235-4b52-a70a-406f28081fb9 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:44.913899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9b258492-c235-4b52-a70a-406f28081fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:44.974308 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1422/2837] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:44.984385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:44.986029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:45.052910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4042752-d04f-4c04-9b60-17c3f2ae3122 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:45.057154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425c3a8-ede6-449b-a923-6b758e59e8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:45.057841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425c3a8-ede6-449b-a923-6b758e59e8dc service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:45.058083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425c3a8-ede6-449b-a923-6b758e59e8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:45.058383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425c3a8-ede6-449b-a923-6b758e59e8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:45.068855 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425c3a8-ede6-449b-a923-6b758e59e8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:45.070417 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425c3a8-ede6-449b-a923-6b758e59e8dc service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:45.081669 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1416/2838] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:45.367673 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b258492-c235-4b52-a70a-406f28081fb9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:45.369146 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9b258492-c235-4b52-a70a-406f28081fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:45.383282 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b258492-c235-4b52-a70a-406f28081fb9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:45.388909 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:45.389653 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:45.390030 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:45.390443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] PATH_INFO: `/v3/projects/cfacb84cbd974aeaa8b9c3d8705e9a0f` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:45.391738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfacb84cbd974aeaa8b9c3d8705e9a0f)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:45.393876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:45.443163 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfacb84cbd974aeaa8b9c3d8705e9a0f', 'request_id': 'req-aefaca7d-1d59-485d-a1e7-92124a926e8e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.449498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfacb84cbd974aeaa8b9c3d8705e9a0f', 'request_id': 'req-aefaca7d-1d59-485d-a1e7-92124a926e8e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.456352 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfacb84cbd974aeaa8b9c3d8705e9a0f', 'request_id': 'req-aefaca7d-1d59-485d-a1e7-92124a926e8e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.470633 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] The token cache is being invalidate because project cfacb84cbd974aeaa8b9c3d8705e9a0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:59:45.471266 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.471614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.473092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1423/2839] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:59:45 2022] DELETE /identity/v3/projects/cfacb84cbd974aeaa8b9c3d8705e9a0f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:45.477477 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425c3a8-ede6-449b-a923-6b758e59e8dc service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:45.487347 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425c3a8-ede6-449b-a923-6b758e59e8dc service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:45.540712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:45.548282 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:45.549151 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:45.549435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:45.549851 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] PATH_INFO: `/v3/users/6c362316ac364a118e063b724a78f70d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:45.551616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c362316ac364a118e063b724a78f70d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:45.553954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:45.602162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c362316ac364a118e063b724a78f70d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.608971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c362316ac364a118e063b724a78f70d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.609627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c362316ac364a118e063b724a78f70d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.618364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c362316ac364a118e063b724a78f70d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.631242 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-ab53b474-1ed0-4377-9dab-b150f626d512 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c362316ac364a118e063b724a78f70d'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:45.637338 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1417/2840] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:59:45 2022] DELETE /identity/v3/users/6c362316ac364a118e063b724a78f70d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:45.918342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:45.919904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:45.935674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefaca7d-1d59-485d-a1e7-92124a926e8e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:45.941461 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:45.995778 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:45.996336 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:45.996709 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:45.996978 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] PATH_INFO: `/v3/projects/5f14a8ca39224b8b86ec66d54954b08e` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:45.998150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f14a8ca39224b8b86ec66d54954b08e)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.000074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.048533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f14a8ca39224b8b86ec66d54954b08e', 'request_id': 'req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:46.054463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f14a8ca39224b8b86ec66d54954b08e', 'request_id': 'req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:46.060800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f14a8ca39224b8b86ec66d54954b08e', 'request_id': 'req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:46.076198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] The token cache is being invalidate because project 5f14a8ca39224b8b86ec66d54954b08e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:59:46.076484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:46.077212 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-a2cf8b75-99b8-4be5-b8d8-a1b998aed16e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:46.078933 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1424/2841] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:59:45 2022] DELETE /identity/v3/projects/5f14a8ca39224b8b86ec66d54954b08e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:46.084223 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3ea45-6100-4492-877a-4a7f56875893 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.084501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3ea45-6100-4492-877a-4a7f56875893 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.084774 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb3ea45-6100-4492-877a-4a7f56875893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.105650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-4bb3ea45-6100-4492-877a-4a7f56875893 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:46.107029 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-4bb3ea45-6100-4492-877a-4a7f56875893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:46.167489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1418/2842] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:59:46.176482 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65cc13-3b86-400e-b63a-fe352782cd26 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.176725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65cc13-3b86-400e-b63a-fe352782cd26 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.176992 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a65cc13-3b86-400e-b63a-fe352782cd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.197217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-2a65cc13-3b86-400e-b63a-fe352782cd26 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:46.198442 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2a65cc13-3b86-400e-b63a-fe352782cd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:46.207044 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1425/2843] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:46.211652 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3ea45-6100-4492-877a-4a7f56875893 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.221468 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb3ea45-6100-4492-877a-4a7f56875893 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.226840 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.227429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.227703 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.227969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.228614 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:59:46.228914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.230536 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.243324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1419/2844] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:59:46.248122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65cc13-3b86-400e-b63a-fe352782cd26 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.256995 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a65cc13-3b86-400e-b63a-fe352782cd26 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.262534 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730bd6a-e073-430a-a986-cb1153db8a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.263128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bd6a-e073-430a-a986-cb1153db8a4e admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.263404 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bd6a-e073-430a-a986-cb1153db8a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.263672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9730bd6a-e073-430a-a986-cb1153db8a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.264306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730bd6a-e073-430a-a986-cb1153db8a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.266275 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9730bd6a-e073-430a-a986-cb1153db8a4e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.297667 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1426/2845] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:59:46 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:46.301869 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.311891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc54f573-d192-442f-ba72-d2012b8def1b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.369759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296515d-1fb3-4be8-9067-dc49aa977b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.370360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296515d-1fb3-4be8-9067-dc49aa977b8f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.370635 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296515d-1fb3-4be8-9067-dc49aa977b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.370885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f296515d-1fb3-4be8-9067-dc49aa977b8f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.371586 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296515d-1fb3-4be8-9067-dc49aa977b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.373506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f296515d-1fb3-4be8-9067-dc49aa977b8f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.383627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:46.397102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1420/2846] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:59:46 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:46.401430 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730bd6a-e073-430a-a986-cb1153db8a4e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.412405 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9730bd6a-e073-430a-a986-cb1153db8a4e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.418218 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33112-271d-4116-a8cb-b3f6400a2cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.418887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33112-271d-4116-a8cb-b3f6400a2cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.419162 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33112-271d-4116-a8cb-b3f6400a2cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.419429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d33112-271d-4116-a8cb-b3f6400a2cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.420069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33112-271d-4116-a8cb-b3f6400a2cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.421918 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d33112-271d-4116-a8cb-b3f6400a2cd6 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.432387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1427/2847] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:59:46.436582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296515d-1fb3-4be8-9067-dc49aa977b8f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.447135 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f296515d-1fb3-4be8-9067-dc49aa977b8f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.452906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c3ee0-2d83-4f87-9fc2-8c10ed0076e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.453535 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3ee0-2d83-4f87-9fc2-8c10ed0076e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.453852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3ee0-2d83-4f87-9fc2-8c10ed0076e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.454120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c3ee0-2d83-4f87-9fc2-8c10ed0076e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.454764 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3ee0-2d83-4f87-9fc2-8c10ed0076e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.456364 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c3ee0-2d83-4f87-9fc2-8c10ed0076e1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.468537 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1421/2848] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:46.472878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33112-271d-4116-a8cb-b3f6400a2cd6 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.483782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d33112-271d-4116-a8cb-b3f6400a2cd6 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.489464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a1503-391c-4f6b-af18-b5ac23befc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.490187 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a1503-391c-4f6b-af18-b5ac23befc39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.490444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a1503-391c-4f6b-af18-b5ac23befc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.490714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a1503-391c-4f6b-af18-b5ac23befc39 admin admin] PATH_INFO: `/v3/projects/22f25246dc6a4328adc78429b7e8f226/users/24b3eddbf7dc47be8ff58b23d008d835/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.506583 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a1503-391c-4f6b-af18-b5ac23befc39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22f25246dc6a4328adc78429b7e8f226, user_id=24b3eddbf7dc47be8ff58b23d008d835, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.509189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119a1503-391c-4f6b-af18-b5ac23befc39 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.525394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1428/2849] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:46 2022] PUT /identity/v3/projects/22f25246dc6a4328adc78429b7e8f226/users/24b3eddbf7dc47be8ff58b23d008d835/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:46.536311 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d685b-e378-412f-b4a9-d2d4413a9102 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.536859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d685b-e378-412f-b4a9-d2d4413a9102 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.536859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d685b-e378-412f-b4a9-d2d4413a9102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.566426 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c71d685b-e378-412f-b4a9-d2d4413a9102 None None] MFA Rules not processed for user `24b3eddbf7dc47be8ff58b23d008d835`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:46.567633 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c71d685b-e378-412f-b4a9-d2d4413a9102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:46.637482 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1422/2850] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:46.643300 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a1503-391c-4f6b-af18-b5ac23befc39 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.695514 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:46.720876 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a1503-391c-4f6b-af18-b5ac23befc39 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.726115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8c8ba-694c-4839-b8e1-e596a77a0c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.726752 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8c8ba-694c-4839-b8e1-e596a77a0c8b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.727043 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8c8ba-694c-4839-b8e1-e596a77a0c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.727304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8c8ba-694c-4839-b8e1-e596a77a0c8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.727925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8c8ba-694c-4839-b8e1-e596a77a0c8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.729899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8c8ba-694c-4839-b8e1-e596a77a0c8b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.757798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1429/2851] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:59:46 2022] POST /identity/v3/projects => generated 383 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:46.761959 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d685b-e378-412f-b4a9-d2d4413a9102 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.772937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d685b-e378-412f-b4a9-d2d4413a9102 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.836715 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd276-3437-4bd2-88d1-af28c32e58dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.837306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd276-3437-4bd2-88d1-af28c32e58dd admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.837582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd276-3437-4bd2-88d1-af28c32e58dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.837899 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfd276-3437-4bd2-88d1-af28c32e58dd admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.838525 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfd276-3437-4bd2-88d1-af28c32e58dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.840467 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cfd276-3437-4bd2-88d1-af28c32e58dd admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.863081 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1423/2852] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:59:46 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 14:59:46.867236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8c8ba-694c-4839-b8e1-e596a77a0c8b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.878152 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8c8ba-694c-4839-b8e1-e596a77a0c8b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.883258 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a9c1b-dc92-43a5-87f1-13534618637d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.883860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9c1b-dc92-43a5-87f1-13534618637d admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.884138 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9c1b-dc92-43a5-87f1-13534618637d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.884498 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a9c1b-dc92-43a5-87f1-13534618637d admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.885165 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a9c1b-dc92-43a5-87f1-13534618637d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.886806 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a9c1b-dc92-43a5-87f1-13534618637d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.897744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1430/2853] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:46.901736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd276-3437-4bd2-88d1-af28c32e58dd admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.912075 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfd276-3437-4bd2-88d1-af28c32e58dd admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:46.917750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9906a31-db2a-4461-93af-7cb27e9ff271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:46.918405 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9906a31-db2a-4461-93af-7cb27e9ff271 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:46.918675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9906a31-db2a-4461-93af-7cb27e9ff271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:46.918910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9906a31-db2a-4461-93af-7cb27e9ff271 admin admin] PATH_INFO: `/v3/projects/e402bca436f7453cb60c80fb84ada9b6/users/dd6a0e1be7724d54a8ee0d0f94458116/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:46.944030 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9906a31-db2a-4461-93af-7cb27e9ff271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e402bca436f7453cb60c80fb84ada9b6, user_id=dd6a0e1be7724d54a8ee0d0f94458116, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:46.946793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9906a31-db2a-4461-93af-7cb27e9ff271 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:46.963698 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1424/2854] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:46 2022] PUT /identity/v3/projects/e402bca436f7453cb60c80fb84ada9b6/users/dd6a0e1be7724d54a8ee0d0f94458116/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 14:59:46.968463 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a9c1b-dc92-43a5-87f1-13534618637d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:46.979420 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a9c1b-dc92-43a5-87f1-13534618637d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:47.043399 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4740d9be-6b15-4e24-8116-722fa538b437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:47.044119 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740d9be-6b15-4e24-8116-722fa538b437 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:47.044495 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740d9be-6b15-4e24-8116-722fa538b437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:47.044838 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4740d9be-6b15-4e24-8116-722fa538b437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:47.045481 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740d9be-6b15-4e24-8116-722fa538b437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:47.047120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4740d9be-6b15-4e24-8116-722fa538b437 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:47.056617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1431/2855] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:46 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:47.061004 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9906a31-db2a-4461-93af-7cb27e9ff271 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:47.071487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9906a31-db2a-4461-93af-7cb27e9ff271 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:47.076729 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:47.077372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:47.077664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:47.077958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] PATH_INFO: `/v3/projects/e402bca436f7453cb60c80fb84ada9b6/users/dd6a0e1be7724d54a8ee0d0f94458116/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:47.079937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e402bca436f7453cb60c80fb84ada9b6, user_id=dd6a0e1be7724d54a8ee0d0f94458116, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:47.082616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:47.093548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1425/2856] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:47 2022] PUT /identity/v3/projects/e402bca436f7453cb60c80fb84ada9b6/users/dd6a0e1be7724d54a8ee0d0f94458116/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:47.104997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d7bf6-cd6b-4062-b160-1721061f5e59 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:47.105306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d7bf6-cd6b-4062-b160-1721061f5e59 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:47.105591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d7bf6-cd6b-4062-b160-1721061f5e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:47.134274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6d2d7bf6-cd6b-4062-b160-1721061f5e59 None None] MFA Rules not processed for user `dd6a0e1be7724d54a8ee0d0f94458116`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:47.135492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6d2d7bf6-cd6b-4062-b160-1721061f5e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:47.209625 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1432/2857] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:47.222312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:47.224140 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:47.252701 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:47.286480 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f59f89d-6775-4af6-a359-26a901649c35 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:47.290454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e23121-ee70-4beb-bfb7-f86fada9d95b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:47.291044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e23121-ee70-4beb-bfb7-f86fada9d95b service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:47.291356 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e23121-ee70-4beb-bfb7-f86fada9d95b service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:47.291618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e23121-ee70-4beb-bfb7-f86fada9d95b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:47.300922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e23121-ee70-4beb-bfb7-f86fada9d95b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:47.302314 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e23121-ee70-4beb-bfb7-f86fada9d95b service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:47.311577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1426/2858] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:47.388588 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d7bf6-cd6b-4062-b160-1721061f5e59 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:47.399128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d7bf6-cd6b-4062-b160-1721061f5e59 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:47.403111 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635da0b-a9d0-4918-9154-50be7845b419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:47.403712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635da0b-a9d0-4918-9154-50be7845b419 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:47.403982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635da0b-a9d0-4918-9154-50be7845b419 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:47.404286 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635da0b-a9d0-4918-9154-50be7845b419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:47.433277 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:47.463410 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635da0b-a9d0-4918-9154-50be7845b419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:47.464864 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635da0b-a9d0-4918-9154-50be7845b419 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:47.473796 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1433/2859] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:47.765576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e23121-ee70-4beb-bfb7-f86fada9d95b service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:47.829143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e23121-ee70-4beb-bfb7-f86fada9d95b service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:47.833978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:47.834649 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:47.834903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:47.835168 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] PATH_INFO: `/v3/users/24b3eddbf7dc47be8ff58b23d008d835` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:47.836668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b3eddbf7dc47be8ff58b23d008d835)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:47.838648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:47.886138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b3eddbf7dc47be8ff58b23d008d835'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:47.892926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b3eddbf7dc47be8ff58b23d008d835'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:47.893675 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b3eddbf7dc47be8ff58b23d008d835'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:47.902492 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b3eddbf7dc47be8ff58b23d008d835'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:47.914107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b3eddbf7dc47be8ff58b23d008d835'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:47.919801 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1427/2860] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:59:47 2022] DELETE /identity/v3/users/24b3eddbf7dc47be8ff58b23d008d835 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:47.924429 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce09d0-6515-4584-ae09-dafa59484578 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:47.924710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce09d0-6515-4584-ae09-dafa59484578 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:47.924983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dce09d0-6515-4584-ae09-dafa59484578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:47.944174 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-3dce09d0-6515-4584-ae09-dafa59484578 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:47.945357 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3dce09d0-6515-4584-ae09-dafa59484578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:48.015396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1434/2861] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:48.025332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:48.026997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:48.079401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:48.085188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24b993-d8fc-4d02-a281-9a635c664bf1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:48.090575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49109f-0b8b-4325-b90f-a18289fd8e35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:48.091294 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49109f-0b8b-4325-b90f-a18289fd8e35 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:48.091610 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49109f-0b8b-4325-b90f-a18289fd8e35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:48.091887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b49109f-0b8b-4325-b90f-a18289fd8e35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:48.101392 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49109f-0b8b-4325-b90f-a18289fd8e35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:48.102857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b49109f-0b8b-4325-b90f-a18289fd8e35 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:48.112752 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1428/2862] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:48.389125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce09d0-6515-4584-ae09-dafa59484578 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:48.390861 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3dce09d0-6515-4584-ae09-dafa59484578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:48.405925 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dce09d0-6515-4584-ae09-dafa59484578 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:48.410917 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:48.411568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:48.411844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:48.412599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] PATH_INFO: `/v3/projects/22f25246dc6a4328adc78429b7e8f226` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:48.414234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=22f25246dc6a4328adc78429b7e8f226)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:48.416367 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:48.464478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22f25246dc6a4328adc78429b7e8f226', 'request_id': 'req-d234c171-f862-4954-b752-f9aaf41f54ce'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.470797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22f25246dc6a4328adc78429b7e8f226', 'request_id': 'req-d234c171-f862-4954-b752-f9aaf41f54ce'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.477097 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22f25246dc6a4328adc78429b7e8f226', 'request_id': 'req-d234c171-f862-4954-b752-f9aaf41f54ce'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.491280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] The token cache is being invalidate because project 22f25246dc6a4328adc78429b7e8f226 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:59:48.491571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.492267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.493934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1435/2863] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:59:48 2022] DELETE /identity/v3/projects/22f25246dc6a4328adc78429b7e8f226 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:48.498176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49109f-0b8b-4325-b90f-a18289fd8e35 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:48.508101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b49109f-0b8b-4325-b90f-a18289fd8e35 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:48.567102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:48.567724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:48.567998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:48.568255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] PATH_INFO: `/v3/users/dd6a0e1be7724d54a8ee0d0f94458116` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:48.569411 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd6a0e1be7724d54a8ee0d0f94458116)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:48.571385 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:48.627112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd6a0e1be7724d54a8ee0d0f94458116'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.634626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd6a0e1be7724d54a8ee0d0f94458116'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.635258 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd6a0e1be7724d54a8ee0d0f94458116'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.644495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd6a0e1be7724d54a8ee0d0f94458116'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.656573 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-3cf9e2fa-eb98-4959-9f20-687210676552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd6a0e1be7724d54a8ee0d0f94458116'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:48.662430 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1429/2864] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 14:59:48 2022] DELETE /identity/v3/users/dd6a0e1be7724d54a8ee0d0f94458116 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:48.943712 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:48.945370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:48.962122 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d234c171-f862-4954-b752-f9aaf41f54ce admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:49.024646 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:49.035446 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:49.036334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.036621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.037051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] PATH_INFO: `/v3/projects/e402bca436f7453cb60c80fb84ada9b6` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.038339 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e402bca436f7453cb60c80fb84ada9b6)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:49.040366 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:49.089508 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e402bca436f7453cb60c80fb84ada9b6', 'request_id': 'req-8756b36d-92cc-42df-9aa6-ae8fd88278c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:49.095603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e402bca436f7453cb60c80fb84ada9b6', 'request_id': 'req-8756b36d-92cc-42df-9aa6-ae8fd88278c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:49.101922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e402bca436f7453cb60c80fb84ada9b6', 'request_id': 'req-8756b36d-92cc-42df-9aa6-ae8fd88278c8'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:49.115707 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] The token cache is being invalidate because project e402bca436f7453cb60c80fb84ada9b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 14:59:49.115994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:49.117091 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8756b36d-92cc-42df-9aa6-ae8fd88278c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 14:59:49.118672 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1436/2865] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 14:59:48 2022] DELETE /identity/v3/projects/e402bca436f7453cb60c80fb84ada9b6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:49.123743 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcbd38-6973-4717-9fa1-a9c3fe7951b7 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.124249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcbd38-6973-4717-9fa1-a9c3fe7951b7 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.124249 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdcbd38-6973-4717-9fa1-a9c3fe7951b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.144260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-3fdcbd38-6973-4717-9fa1-a9c3fe7951b7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:49.145471 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-3fdcbd38-6973-4717-9fa1-a9c3fe7951b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:49.215532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1430/2866] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:49.224037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f7cb2-463e-4e00-9f86-ea1ec37436d0 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.224392 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f7cb2-463e-4e00-9f86-ea1ec37436d0 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.224737 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1f7cb2-463e-4e00-9f86-ea1ec37436d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.247809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6f1f7cb2-463e-4e00-9f86-ea1ec37436d0 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:49.249027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6f1f7cb2-463e-4e00-9f86-ea1ec37436d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:49.257189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1437/2867] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 14:59:49.261522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcbd38-6973-4717-9fa1-a9c3fe7951b7 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:49.272563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdcbd38-6973-4717-9fa1-a9c3fe7951b7 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:49.280544 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:49.281390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.281778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.282167 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.283142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 14:59:49.283549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:49.286084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:49.289762 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:49.301766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1431/2868] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 14:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 14:59:49.306340 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f7cb2-463e-4e00-9f86-ea1ec37436d0 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:49.316164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1f7cb2-463e-4e00-9f86-ea1ec37436d0 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:49.322219 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ad1b7-f39c-44fa-b211-fd06f777f84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:49.322849 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ad1b7-f39c-44fa-b211-fd06f777f84b admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.323121 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ad1b7-f39c-44fa-b211-fd06f777f84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.323384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ad1b7-f39c-44fa-b211-fd06f777f84b admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.324033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ad1b7-f39c-44fa-b211-fd06f777f84b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:49.326238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ad1b7-f39c-44fa-b211-fd06f777f84b admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:49.369048 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1438/2869] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 14:59:49 2022] POST /identity/v3/projects => generated 369 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:49.373903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:49.389897 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee548f-581b-4ed2-86fe-53c2e5435428 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:49.457853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c554ff3-6b6d-446f-8ceb-f4875031c683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:49.459056 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c554ff3-6b6d-446f-8ceb-f4875031c683 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.459358 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c554ff3-6b6d-446f-8ceb-f4875031c683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.459628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c554ff3-6b6d-446f-8ceb-f4875031c683 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.460257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c554ff3-6b6d-446f-8ceb-f4875031c683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:49.462627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c554ff3-6b6d-446f-8ceb-f4875031c683 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:49.488186 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1432/2870] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 14:59:49 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 14:59:49.493954 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ad1b7-f39c-44fa-b211-fd06f777f84b admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:49.506604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ad1b7-f39c-44fa-b211-fd06f777f84b admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:49.511695 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dd0be-e09f-46bc-ac0e-0d2fba72a258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:49.512739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dd0be-e09f-46bc-ac0e-0d2fba72a258 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.512739 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dd0be-e09f-46bc-ac0e-0d2fba72a258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.512946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909dd0be-e09f-46bc-ac0e-0d2fba72a258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.513437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dd0be-e09f-46bc-ac0e-0d2fba72a258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:49.515075 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909dd0be-e09f-46bc-ac0e-0d2fba72a258 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:49.525151 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1439/2871] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 14:59:49.529383 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c554ff3-6b6d-446f-8ceb-f4875031c683 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:49.539843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c554ff3-6b6d-446f-8ceb-f4875031c683 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:49.545108 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a4d9a3-fa47-4a8c-82fa-9e433b694d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:49.545734 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4d9a3-fa47-4a8c-82fa-9e433b694d95 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.546010 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4d9a3-fa47-4a8c-82fa-9e433b694d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.546277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a4d9a3-fa47-4a8c-82fa-9e433b694d95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.546925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4d9a3-fa47-4a8c-82fa-9e433b694d95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:49.548507 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a4d9a3-fa47-4a8c-82fa-9e433b694d95 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:49.559197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1433/2872] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 14:59:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 14:59:49.563242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dd0be-e09f-46bc-ac0e-0d2fba72a258 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:49.573397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909dd0be-e09f-46bc-ac0e-0d2fba72a258 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:49.578668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:49.579304 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.579579 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.579880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] PATH_INFO: `/v3/projects/e40a1ba97caa454794ec2e4234ef379b/users/47a8ccf7763544199357001897612e07/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.596238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e40a1ba97caa454794ec2e4234ef379b, user_id=47a8ccf7763544199357001897612e07, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:49.599037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:49.612844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1440/2873] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 14:59:49 2022] PUT /identity/v3/projects/e40a1ba97caa454794ec2e4234ef379b/users/47a8ccf7763544199357001897612e07/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 14:59:49.623275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b546c-07ca-471e-a1e9-0cbb1f221e46 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.623556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b546c-07ca-471e-a1e9-0cbb1f221e46 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.623854 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623b546c-07ca-471e-a1e9-0cbb1f221e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.655003 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-623b546c-07ca-471e-a1e9-0cbb1f221e46 None None] MFA Rules not processed for user `47a8ccf7763544199357001897612e07`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 14:59:49.656217 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-623b546c-07ca-471e-a1e9-0cbb1f221e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:49.728522 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1434/2874] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 14:59:49 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 14:59:49.740522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 14:59:49.742799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 14:59:49.802755 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5553b8d-6f3b-4032-ac4d-1d7f14059137 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 14:59:49.807671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfd2cd-dd07-4752-8bce-6f5ae2ae0903 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417b661f770846f189c36d0804b6dce5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 14:59:49.808306 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfd2cd-dd07-4752-8bce-6f5ae2ae0903 service glance] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 14:59:49.808573 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfd2cd-dd07-4752-8bce-6f5ae2ae0903 service glance] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 14:59:49.808830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfd2cd-dd07-4752-8bce-6f5ae2ae0903 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 14:59:49.818253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfd2cd-dd07-4752-8bce-6f5ae2ae0903 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 14:59:49.819711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfd2cd-dd07-4752-8bce-6f5ae2ae0903 service glance] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 14:59:49.829206 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1441/2875] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 14:59:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 14:59:59.725389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=68,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 14:59:59.824797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:00:54.312979 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b546c-07ca-471e-a1e9-0cbb1f221e46 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:54.381697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623b546c-07ca-471e-a1e9-0cbb1f221e46 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:54.387283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:54.387931 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:54.388208 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:54.388501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] PATH_INFO: `/v3/users/47a8ccf7763544199357001897612e07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:54.389778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=47a8ccf7763544199357001897612e07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:54.391812 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:54.445300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47a8ccf7763544199357001897612e07'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:54.452712 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47a8ccf7763544199357001897612e07'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:54.453421 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47a8ccf7763544199357001897612e07'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:54.462456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47a8ccf7763544199357001897612e07'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:54.477293 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47a8ccf7763544199357001897612e07'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:54.483272 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1435/2876] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:00:54 2022] DELETE /identity/v3/users/47a8ccf7763544199357001897612e07 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:00:54.487798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a2ff5-29c2-450c-a85d-07e8afa2331c None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:54.488353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a2ff5-29c2-450c-a85d-07e8afa2331c None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:54.488353 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2a2ff5-29c2-450c-a85d-07e8afa2331c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:54.508881 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1d2a2ff5-29c2-450c-a85d-07e8afa2331c None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:00:54.510260 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1d2a2ff5-29c2-450c-a85d-07e8afa2331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:54.571387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1442/2877] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:00:54.581321 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:54.582848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:54.645106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db67aec-0bc2-402b-a6f5-fd4637a9a67e admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:54.650005 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08862b6c-87f9-4c9f-a192-0fac8c963c7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:54.650685 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08862b6c-87f9-4c9f-a192-0fac8c963c7b service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:54.650966 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08862b6c-87f9-4c9f-a192-0fac8c963c7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:54.651237 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08862b6c-87f9-4c9f-a192-0fac8c963c7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:54.662416 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08862b6c-87f9-4c9f-a192-0fac8c963c7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:54.663925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08862b6c-87f9-4c9f-a192-0fac8c963c7b service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:54.675088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1436/2878] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:00:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 05 15:00:54.964900 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a2ff5-29c2-450c-a85d-07e8afa2331c None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:54.966478 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1d2a2ff5-29c2-450c-a85d-07e8afa2331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:54.982685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2a2ff5-29c2-450c-a85d-07e8afa2331c None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:54.988729 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:54.989381 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:54.989690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:54.989963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] PATH_INFO: `/v3/projects/e40a1ba97caa454794ec2e4234ef379b` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:54.991417 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e40a1ba97caa454794ec2e4234ef379b)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:54.993394 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:55.041904 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e40a1ba97caa454794ec2e4234ef379b', 'request_id': 'req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:55.047487 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e40a1ba97caa454794ec2e4234ef379b', 'request_id': 'req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:55.053621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e40a1ba97caa454794ec2e4234ef379b', 'request_id': 'req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:55.067985 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] The token cache is being invalidate because project e40a1ba97caa454794ec2e4234ef379b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:00:55.068267 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:55.068893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-5ad5e64e-949b-400e-ac3c-a5323d46ff3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:55.070454 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1443/2879] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:00:54 2022] DELETE /identity/v3/projects/e40a1ba97caa454794ec2e4234ef379b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:00:55.075623 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c824a-0be3-4597-82e7-ba428000597d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.075843 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c824a-0be3-4597-82e7-ba428000597d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.076148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c824a-0be3-4597-82e7-ba428000597d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.095936 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-006c824a-0be3-4597-82e7-ba428000597d None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:00:55.097125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-006c824a-0be3-4597-82e7-ba428000597d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:55.113977 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:00:55.157710 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1437/2880] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:00:55.165814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf6123-4b88-4759-b1fa-4df3e032f46b None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.166066 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf6123-4b88-4759-b1fa-4df3e032f46b None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.166346 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf6123-4b88-4759-b1fa-4df3e032f46b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.186969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-99cf6123-4b88-4759-b1fa-4df3e032f46b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:00:55.188171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-99cf6123-4b88-4759-b1fa-4df3e032f46b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:55.197347 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1444/2881] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:00:55.201798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c824a-0be3-4597-82e7-ba428000597d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:55.212077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c824a-0be3-4597-82e7-ba428000597d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:55.217665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:55.218521 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.218911 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.219289 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.220085 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 15:00:55.220381 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:55.222018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:55.234244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1438/2882] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 15:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 15:00:55.238885 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf6123-4b88-4759-b1fa-4df3e032f46b None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:55.260507 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf6123-4b88-4759-b1fa-4df3e032f46b None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:55.266115 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c193b9-5049-4130-9af5-3d3e3ff2cc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:55.266725 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c193b9-5049-4130-9af5-3d3e3ff2cc41 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.266998 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c193b9-5049-4130-9af5-3d3e3ff2cc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.267265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c193b9-5049-4130-9af5-3d3e3ff2cc41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.267892 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c193b9-5049-4130-9af5-3d3e3ff2cc41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:55.269839 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c193b9-5049-4130-9af5-3d3e3ff2cc41 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:55.300688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1445/2883] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:00:55 2022] POST /identity/v3/projects => generated 363 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:00:55.307534 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:55.319246 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565ebc0-5015-498b-a94e-656bbff90226 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:55.380530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c90277-a748-45d5-b49a-a818deb3f17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:55.381082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90277-a748-45d5-b49a-a818deb3f17d admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.381354 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90277-a748-45d5-b49a-a818deb3f17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.381664 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c90277-a748-45d5-b49a-a818deb3f17d admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.382287 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c90277-a748-45d5-b49a-a818deb3f17d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:55.384226 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c90277-a748-45d5-b49a-a818deb3f17d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:55.409431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1439/2884] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:00:55 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:00:55.415360 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c193b9-5049-4130-9af5-3d3e3ff2cc41 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:55.426149 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c193b9-5049-4130-9af5-3d3e3ff2cc41 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:55.431964 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f9b1-2213-4b3f-a0a8-a6bb7f5807fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:55.432605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f9b1-2213-4b3f-a0a8-a6bb7f5807fe admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.432877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f9b1-2213-4b3f-a0a8-a6bb7f5807fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.433135 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534f9b1-2213-4b3f-a0a8-a6bb7f5807fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.433813 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534f9b1-2213-4b3f-a0a8-a6bb7f5807fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:55.435398 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534f9b1-2213-4b3f-a0a8-a6bb7f5807fe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:55.446141 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1446/2885] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:00:55.450365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c90277-a748-45d5-b49a-a818deb3f17d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:55.461128 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c90277-a748-45d5-b49a-a818deb3f17d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:55.466380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e8af53-8f9c-4daf-a3d9-f01e91c0d692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:55.467123 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8af53-8f9c-4daf-a3d9-f01e91c0d692 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.467447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8af53-8f9c-4daf-a3d9-f01e91c0d692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.467724 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e8af53-8f9c-4daf-a3d9-f01e91c0d692 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.468370 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8af53-8f9c-4daf-a3d9-f01e91c0d692 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:55.472454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e8af53-8f9c-4daf-a3d9-f01e91c0d692 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:55.481659 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1440/2886] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:00:55.486196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f9b1-2213-4b3f-a0a8-a6bb7f5807fe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:55.499923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534f9b1-2213-4b3f-a0a8-a6bb7f5807fe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:55.509009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:55.509009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.509009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.509009 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] PATH_INFO: `/v3/projects/129dbfb444a448a983c52894ddeefe96/users/6b4b7e6090de4d3da1dcb8b45edf5b45/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.525946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=129dbfb444a448a983c52894ddeefe96, user_id=6b4b7e6090de4d3da1dcb8b45edf5b45, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:55.528784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:55.543989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1447/2887] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:00:55 2022] PUT /identity/v3/projects/129dbfb444a448a983c52894ddeefe96/users/6b4b7e6090de4d3da1dcb8b45edf5b45/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:00:55.557146 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58aaa-b04c-47d4-a27d-1b901e696c82 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.557435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58aaa-b04c-47d4-a27d-1b901e696c82 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.557756 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e58aaa-b04c-47d4-a27d-1b901e696c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.592485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-89e58aaa-b04c-47d4-a27d-1b901e696c82 None None] MFA Rules not processed for user `6b4b7e6090de4d3da1dcb8b45edf5b45`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:00:55.593766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89e58aaa-b04c-47d4-a27d-1b901e696c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:55.664730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1441/2888] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:00:55.675888 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:55.677750 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:55.700179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:00:55.739505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7434e9-7f25-49d3-ac4a-3283a8108afe admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:55.743351 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:55.743975 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:55.744276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:55.744513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:55.753728 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:55.755233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:55.764453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1448/2889] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:00:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:00:57.912603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e58aaa-b04c-47d4-a27d-1b901e696c82 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:57.914229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-89e58aaa-b04c-47d4-a27d-1b901e696c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:57.935880 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:00:57.975077 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e58aaa-b04c-47d4-a27d-1b901e696c82 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:57.979143 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:57.979778 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:57.980038 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:57.980292 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:57.982498 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:58.066234 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:58.067661 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:58.077414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1442/2890] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:00:58.595073 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:58.596586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:00:58.633571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051ecfd1-b721-4695-8fdf-9f949aaf4f32 service cinder] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:58.638297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:58.638923 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:58.639198 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:58.639465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] PATH_INFO: `/v3/users/bc27904d964e43438c32876b8a1b202d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:58.640651 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc27904d964e43438c32876b8a1b202d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:58.642600 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:58.693128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc27904d964e43438c32876b8a1b202d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:58.700748 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc27904d964e43438c32876b8a1b202d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:58.701485 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc27904d964e43438c32876b8a1b202d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:58.709730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc27904d964e43438c32876b8a1b202d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:58.722536 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc27904d964e43438c32876b8a1b202d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:58.728603 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1449/2891] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:00:58 2022] DELETE /identity/v3/users/bc27904d964e43438c32876b8a1b202d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:00:58.914099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:00:58.929819 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47b8e0-6eff-48c9-aae5-9e3935c28bee service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:00:59.005624 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:00:59.006554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:00:59.006910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:00:59.007310 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] PATH_INFO: `/v3/projects/49adc8da67bf4529a349a2bd79e87e07` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:00:59.009026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] RBAC: Authorizing `identity:delete_project(project_id=49adc8da67bf4529a349a2bd79e87e07)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:00:59.010998 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:00:59.065487 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49adc8da67bf4529a349a2bd79e87e07', 'request_id': 'req-db7a4ee0-60b8-4dda-9b59-632fb597904b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:59.066723 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:00:59.072594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49adc8da67bf4529a349a2bd79e87e07', 'request_id': 'req-db7a4ee0-60b8-4dda-9b59-632fb597904b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:59.080668 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49adc8da67bf4529a349a2bd79e87e07', 'request_id': 'req-db7a4ee0-60b8-4dda-9b59-632fb597904b'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:59.095491 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] The token cache is being invalidate because project 49adc8da67bf4529a349a2bd79e87e07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:00:59.095821 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:59.096578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:00:59.098252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1443/2892] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:00:58 2022] DELETE /identity/v3/projects/49adc8da67bf4529a349a2bd79e87e07 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:01:07.202680 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:07.204334 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:07.295714 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:07.298710 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee6b2f-3e69-4de2-b724-e9f13c3ad87e admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:07.361008 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f98649b-d603-481e-a356-c569fafeffee service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:07.361614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98649b-d603-481e-a356-c569fafeffee service placement] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:07.361922 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98649b-d603-481e-a356-c569fafeffee service placement] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:07.362188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f98649b-d603-481e-a356-c569fafeffee service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:07.364088 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8f98649b-d603-481e-a356-c569fafeffee service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:07.375371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98649b-d603-481e-a356-c569fafeffee service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:07.377004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f98649b-d603-481e-a356-c569fafeffee service placement] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:07.387232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1450/2893] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:01:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:01:09.092438 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:17.382897 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=30 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:39.776561 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:39.778195 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:39.846161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a4ee0-60b8-4dda-9b59-632fb597904b admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:39.851205 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:39.851832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:39.852107 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:39.852374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] PATH_INFO: `/v3/users/6b4b7e6090de4d3da1dcb8b45edf5b45` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:39.853529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b4b7e6090de4d3da1dcb8b45edf5b45)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:39.855472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:39.904973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b4b7e6090de4d3da1dcb8b45edf5b45'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:39.912070 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b4b7e6090de4d3da1dcb8b45edf5b45'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:39.912732 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b4b7e6090de4d3da1dcb8b45edf5b45'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:39.922969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b4b7e6090de4d3da1dcb8b45edf5b45'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:39.935463 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-32c38497-3961-4df7-9e3e-d71c6f82b3ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b4b7e6090de4d3da1dcb8b45edf5b45'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:39.941419 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1444/2894] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:01:39 2022] DELETE /identity/v3/users/6b4b7e6090de4d3da1dcb8b45edf5b45 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:01:39.946184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130e5d-4e59-45d2-93e1-cc17d2e2aced None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:39.946705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130e5d-4e59-45d2-93e1-cc17d2e2aced None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:39.946705 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130e5d-4e59-45d2-93e1-cc17d2e2aced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:39.966361 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-b4130e5d-4e59-45d2-93e1-cc17d2e2aced None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:01:39.967569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-b4130e5d-4e59-45d2-93e1-cc17d2e2aced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:40.027063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1451/2895] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:01:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:01:40.037161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cad78-a0a4-4795-9c6a-dbfaebfd1bea None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.037506 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cad78-a0a4-4795-9c6a-dbfaebfd1bea None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.037926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343cad78-a0a4-4795-9c6a-dbfaebfd1bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.081125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-343cad78-a0a4-4795-9c6a-dbfaebfd1bea None None] MFA Rules not processed for user `6c9b940cd9e0432daa235f82becb4d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:01:40.082365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-343cad78-a0a4-4795-9c6a-dbfaebfd1bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:40.144346 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1445/2896] 10.222.0.23 () {58 vars in 1061 bytes} [Wed Oct 5 15:01:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:01:40.151064 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4130e5d-4e59-45d2-93e1-cc17d2e2aced None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:40.162051 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4130e5d-4e59-45d2-93e1-cc17d2e2aced None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:40.166556 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e705785-b77b-4aae-a44b-e4189e1664e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:40.167170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e705785-b77b-4aae-a44b-e4189e1664e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.167453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e705785-b77b-4aae-a44b-e4189e1664e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.167722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e705785-b77b-4aae-a44b-e4189e1664e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.177170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e705785-b77b-4aae-a44b-e4189e1664e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:40.178745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e705785-b77b-4aae-a44b-e4189e1664e9 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:40.188386 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1452/2897] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:01:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:01:40.457852 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cad78-a0a4-4795-9c6a-dbfaebfd1bea None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:40.459435 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-343cad78-a0a4-4795-9c6a-dbfaebfd1bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:40.477160 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343cad78-a0a4-4795-9c6a-dbfaebfd1bea None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:40.483437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:40.484088 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.484357 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.484695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] PATH_INFO: `/v3/projects/129dbfb444a448a983c52894ddeefe96` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.485923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=129dbfb444a448a983c52894ddeefe96)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:40.487884 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:40.499622 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:40.541497 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '129dbfb444a448a983c52894ddeefe96', 'request_id': 'req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:40.547363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '129dbfb444a448a983c52894ddeefe96', 'request_id': 'req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:40.553725 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '129dbfb444a448a983c52894ddeefe96', 'request_id': 'req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:40.574628 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] The token cache is being invalidate because project 129dbfb444a448a983c52894ddeefe96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:01:40.574895 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:40.575530 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-5676f985-f096-4d0b-8b8e-3e9a8413f7e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:40.577052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1446/2898] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:01:40 2022] DELETE /identity/v3/projects/129dbfb444a448a983c52894ddeefe96 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:01:40.582203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34e78-975b-4992-aece-c8bd4f924705 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.582483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34e78-975b-4992-aece-c8bd4f924705 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.582751 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce34e78-975b-4992-aece-c8bd4f924705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.603320 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-8ce34e78-975b-4992-aece-c8bd4f924705 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:01:40.604529 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-8ce34e78-975b-4992-aece-c8bd4f924705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:40.667704 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1453/2899] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:01:40.675978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5779e55-c29d-4538-b314-24285d8b3de1 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.676259 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5779e55-c29d-4538-b314-24285d8b3de1 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.676533 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5779e55-c29d-4538-b314-24285d8b3de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.698169 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-d5779e55-c29d-4538-b314-24285d8b3de1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:01:40.699493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-d5779e55-c29d-4538-b314-24285d8b3de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:40.711241 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1447/2900] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:01:40.718415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce34e78-975b-4992-aece-c8bd4f924705 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:40.732562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce34e78-975b-4992-aece-c8bd4f924705 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:40.738059 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:40.738657 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.738928 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.739189 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.739824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 15:01:40.740120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:40.741732 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:40.753979 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1454/2901] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 15:01:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 15:01:40.758375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5779e55-c29d-4538-b314-24285d8b3de1 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:40.767746 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5779e55-c29d-4538-b314-24285d8b3de1 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:40.772885 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6627-a2a2-4ba0-a55f-db8412a8b479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:40.773479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6627-a2a2-4ba0-a55f-db8412a8b479 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.773782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6627-a2a2-4ba0-a55f-db8412a8b479 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.774050 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6627-a2a2-4ba0-a55f-db8412a8b479 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.774679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e6627-a2a2-4ba0-a55f-db8412a8b479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:40.776566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e6627-a2a2-4ba0-a55f-db8412a8b479 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:40.804937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1448/2902] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:01:40 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:01:40.809240 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:40.819965 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c90054-6309-4076-a6f1-2d66c15e57ca admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:40.879533 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4188f-6876-4c02-bafc-1dab20c39606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:40.880142 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4188f-6876-4c02-bafc-1dab20c39606 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.880419 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4188f-6876-4c02-bafc-1dab20c39606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.880688 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e4188f-6876-4c02-bafc-1dab20c39606 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.881312 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4188f-6876-4c02-bafc-1dab20c39606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:40.883387 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e4188f-6876-4c02-bafc-1dab20c39606 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:40.909183 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1455/2903] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:01:40 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:01:40.910437 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6627-a2a2-4ba0-a55f-db8412a8b479 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:40.923117 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6627-a2a2-4ba0-a55f-db8412a8b479 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:40.928589 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abe224-1450-422b-8c41-bc5e22abada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:40.929213 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abe224-1450-422b-8c41-bc5e22abada2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.929489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abe224-1450-422b-8c41-bc5e22abada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.929797 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99abe224-1450-422b-8c41-bc5e22abada2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.930473 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abe224-1450-422b-8c41-bc5e22abada2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:40.932093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99abe224-1450-422b-8c41-bc5e22abada2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:40.943257 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1449/2904] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:01:40.947643 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4188f-6876-4c02-bafc-1dab20c39606 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:40.954194 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:40.958025 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e4188f-6876-4c02-bafc-1dab20c39606 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:40.964045 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955c5a2f-afd9-4eeb-bcf4-1a42b09474ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:40.964840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5a2f-afd9-4eeb-bcf4-1a42b09474ea admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:40.965295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5a2f-afd9-4eeb-bcf4-1a42b09474ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:40.965718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955c5a2f-afd9-4eeb-bcf4-1a42b09474ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:40.966541 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c5a2f-afd9-4eeb-bcf4-1a42b09474ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:40.968298 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955c5a2f-afd9-4eeb-bcf4-1a42b09474ea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:40.979441 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1456/2905] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:01:40.983670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abe224-1450-422b-8c41-bc5e22abada2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:40.993958 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99abe224-1450-422b-8c41-bc5e22abada2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:40.999342 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:41.000016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:41.000286 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:41.000552 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] PATH_INFO: `/v3/projects/26c46426bcea4d55bb6d67174cc7bdb2/users/696e3414f70646d7a042cec0e3a89524/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:41.015741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] RBAC: Authorizing `identity:create_grant(project_id=26c46426bcea4d55bb6d67174cc7bdb2, user_id=696e3414f70646d7a042cec0e3a89524, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:41.018457 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:41.035401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1450/2906] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:01:40 2022] PUT /identity/v3/projects/26c46426bcea4d55bb6d67174cc7bdb2/users/696e3414f70646d7a042cec0e3a89524/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 15:01:41.046037 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e9365-a6ea-48d9-ba51-9cff5e8abfa6 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:41.046315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e9365-a6ea-48d9-ba51-9cff5e8abfa6 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:41.047196 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65e9365-a6ea-48d9-ba51-9cff5e8abfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:41.074685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-e65e9365-a6ea-48d9-ba51-9cff5e8abfa6 None None] MFA Rules not processed for user `696e3414f70646d7a042cec0e3a89524`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:01:41.075811 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-e65e9365-a6ea-48d9-ba51-9cff5e8abfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:41.141005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1457/2907] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:01:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:01:41.151796 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:41.153833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:41.218459 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abed2bf-3715-495a-8ee1-a8f12abab28d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:41.224103 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bde102-d93a-469f-aa1f-e605ace74478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:41.225019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bde102-d93a-469f-aa1f-e605ace74478 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:41.225366 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bde102-d93a-469f-aa1f-e605ace74478 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:41.225709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bde102-d93a-469f-aa1f-e605ace74478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:41.237197 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bde102-d93a-469f-aa1f-e605ace74478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:41.238705 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bde102-d93a-469f-aa1f-e605ace74478 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:41.248744 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1451/2908] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:01:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:01:51.138443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=35 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:51.244304 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:51.550489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e9365-a6ea-48d9-ba51-9cff5e8abfa6 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:51.623249 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65e9365-a6ea-48d9-ba51-9cff5e8abfa6 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:51.629407 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:51.630145 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:51.630426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:51.630693 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] PATH_INFO: `/v3/users/696e3414f70646d7a042cec0e3a89524` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:51.631850 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] RBAC: Authorizing `identity:delete_user(user_id=696e3414f70646d7a042cec0e3a89524)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:51.633983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:51.683094 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696e3414f70646d7a042cec0e3a89524'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:51.689853 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696e3414f70646d7a042cec0e3a89524'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:51.690567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696e3414f70646d7a042cec0e3a89524'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:51.699722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696e3414f70646d7a042cec0e3a89524'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:51.711594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8cdc8000-4436-432b-a6fc-10752fc62860 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696e3414f70646d7a042cec0e3a89524'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:51.717866 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1458/2909] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:01:51 2022] DELETE /identity/v3/users/696e3414f70646d7a042cec0e3a89524 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:01:51.724053 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986ce9f-08e4-4b53-9f9b-acd7bd4b4199 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:51.724328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986ce9f-08e4-4b53-9f9b-acd7bd4b4199 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:51.724593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986ce9f-08e4-4b53-9f9b-acd7bd4b4199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:51.744479 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f986ce9f-08e4-4b53-9f9b-acd7bd4b4199 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:01:51.745677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f986ce9f-08e4-4b53-9f9b-acd7bd4b4199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:51.805201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1452/2910] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:01:51.814777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06bd16-95eb-4287-8dde-f43cd8d8c4a2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:51.815057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06bd16-95eb-4287-8dde-f43cd8d8c4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:51.815323 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea06bd16-95eb-4287-8dde-f43cd8d8c4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:51.835699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ea06bd16-95eb-4287-8dde-f43cd8d8c4a2 None None] MFA Rules not processed for user `6c9b940cd9e0432daa235f82becb4d38`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:01:51.836908 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea06bd16-95eb-4287-8dde-f43cd8d8c4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:51.890862 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1459/2911] 10.222.0.23 () {58 vars in 1061 bytes} [Wed Oct 5 15:01:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:01:51.896822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986ce9f-08e4-4b53-9f9b-acd7bd4b4199 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:51.907104 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986ce9f-08e4-4b53-9f9b-acd7bd4b4199 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:51.911134 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:51.911737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:51.912026 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:51.912291 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:51.922665 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:51.924101 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:51.934093 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1453/2912] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:01:52.218665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06bd16-95eb-4287-8dde-f43cd8d8c4a2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:52.220371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ea06bd16-95eb-4287-8dde-f43cd8d8c4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:52.235966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea06bd16-95eb-4287-8dde-f43cd8d8c4a2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:52.240860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:52.241504 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:52.241805 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:52.242182 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] PATH_INFO: `/v3/projects/26c46426bcea4d55bb6d67174cc7bdb2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:52.243261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] RBAC: Authorizing `identity:delete_project(project_id=26c46426bcea4d55bb6d67174cc7bdb2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:52.245117 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:52.271715 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:52.298011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26c46426bcea4d55bb6d67174cc7bdb2', 'request_id': 'req-3d927223-950b-4916-a853-86cda0b142df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:52.303544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26c46426bcea4d55bb6d67174cc7bdb2', 'request_id': 'req-3d927223-950b-4916-a853-86cda0b142df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:52.309987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26c46426bcea4d55bb6d67174cc7bdb2', 'request_id': 'req-3d927223-950b-4916-a853-86cda0b142df'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:52.324233 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] The token cache is being invalidate because project 26c46426bcea4d55bb6d67174cc7bdb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:01:52.324432 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:52.325033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:01:52.326550 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1460/2913] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:01:52 2022] DELETE /identity/v3/projects/26c46426bcea4d55bb6d67174cc7bdb2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:01:52.642954 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:01:52.644853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:52.726142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5425a036-4524-4162-b45c-bb5f422c830d service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:01:52.792470 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83f4897f360f4f23aac5749685fdf500', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:01:52.793072 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:01:52.793350 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:01:52.793709 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:01:52.795515 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:01:52.817224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=100 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:01:52.863017 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:01:52.864626 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:01:52.874047 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1454/2914] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:01:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:02.320983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:02.869971 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=17 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:24.495116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:24.496531 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:24.557645 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d927223-950b-4916-a853-86cda0b142df admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:24.561877 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:24.562513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:24.562787 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:24.563053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:24.565010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:24.643401 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:24.644777 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:24.654848 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1461/2915] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:27.825428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:27.830663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:27.901944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:27.903375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:27.914983 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fa9ba-fde8-4cee-83cb-2ea57eb4889f service nova] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:27.920772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ac0bdd-9d09-4021-9899-bf28217757b0 service neutron] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:27.922848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:27.923485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:27.923760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:27.924133 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:27.925577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:27.925891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:27.926359 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:27.926717 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:27.927071 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] PATH_INFO: `/v3/users/7de0d01f891d4d73bc44810aacbca477` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:27.928170 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7de0d01f891d4d73bc44810aacbca477)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:27.930384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:27.938373 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:27.940083 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:27.952407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1455/2916] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:27.982125 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7de0d01f891d4d73bc44810aacbca477'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:27.991986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7de0d01f891d4d73bc44810aacbca477'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:27.992685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7de0d01f891d4d73bc44810aacbca477'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:28.000204 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7de0d01f891d4d73bc44810aacbca477'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:28.012120 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-c998b086-de3c-49c6-a907-3d8912cf8b11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7de0d01f891d4d73bc44810aacbca477'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:28.018014 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1462/2917] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:02:27 2022] DELETE /identity/v3/users/7de0d01f891d4d73bc44810aacbca477 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:28.190434 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:28.203961 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa24749-ce0f-40cd-824b-e8a8972baa06 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:28.265251 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:28.265922 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.266222 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.266494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] PATH_INFO: `/v3/projects/ff81348c32db4e40a8e331317d14e3d2` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.267640 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff81348c32db4e40a8e331317d14e3d2)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:28.269553 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:28.318940 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff81348c32db4e40a8e331317d14e3d2', 'request_id': 'req-61529487-b812-4ee4-93d6-97373cd0650f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:28.324713 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff81348c32db4e40a8e331317d14e3d2', 'request_id': 'req-61529487-b812-4ee4-93d6-97373cd0650f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:28.330948 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff81348c32db4e40a8e331317d14e3d2', 'request_id': 'req-61529487-b812-4ee4-93d6-97373cd0650f'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:28.345139 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] The token cache is being invalidate because project ff81348c32db4e40a8e331317d14e3d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81571) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:02:28.345428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:28.346065 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-61529487-b812-4ee4-93d6-97373cd0650f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:28.347577 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1456/2918] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:02:28 2022] DELETE /identity/v3/projects/ff81348c32db4e40a8e331317d14e3d2 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:28.352738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0440-5275-4edb-822a-ff05636592bf None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.353023 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0440-5275-4edb-822a-ff05636592bf None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.353296 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c0440-5275-4edb-822a-ff05636592bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.372981 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-9f0c0440-5275-4edb-822a-ff05636592bf None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:28.374220 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-9f0c0440-5275-4edb-822a-ff05636592bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:28.391317 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:28.435396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1463/2919] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:28.443737 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936c628-54f5-44ab-b130-cac650262156 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.444116 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936c628-54f5-44ab-b130-cac650262156 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.444450 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1936c628-54f5-44ab-b130-cac650262156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.469422 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-1936c628-54f5-44ab-b130-cac650262156 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:28.470695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1936c628-54f5-44ab-b130-cac650262156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:28.479944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1457/2920] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:28.484425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c0440-5275-4edb-822a-ff05636592bf None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:28.494575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c0440-5275-4edb-822a-ff05636592bf None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:28.499987 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:28.500590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.500865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.501153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] PATH_INFO: `/v3/domains` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.501891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81570) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 15:02:28.502116 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:28.503745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:28.516056 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1464/2921] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 15:02:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 15:02:28.520557 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936c628-54f5-44ab-b130-cac650262156 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:28.530156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1936c628-54f5-44ab-b130-cac650262156 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:28.535129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0b10e-5312-488b-8a1a-e2d02ba43ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:28.535738 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b10e-5312-488b-8a1a-e2d02ba43ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.536014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b10e-5312-488b-8a1a-e2d02ba43ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.536283 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0b10e-5312-488b-8a1a-e2d02ba43ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.536914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0b10e-5312-488b-8a1a-e2d02ba43ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:28.538859 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0b10e-5312-488b-8a1a-e2d02ba43ecf admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:28.550094 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:28.570270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1458/2922] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:02:28 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:28.574477 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:28.584566 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5059ea12-4187-4a0e-85de-5af2330ced86 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:28.646078 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c01d77-a14b-48ec-9359-b495d7036902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:28.646722 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c01d77-a14b-48ec-9359-b495d7036902 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.646994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c01d77-a14b-48ec-9359-b495d7036902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.647261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c01d77-a14b-48ec-9359-b495d7036902 admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.647927 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c01d77-a14b-48ec-9359-b495d7036902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:28.649890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c01d77-a14b-48ec-9359-b495d7036902 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:28.674180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1465/2923] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:02:28 2022] POST /identity/v3/users => generated 290 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 15:02:28.680309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0b10e-5312-488b-8a1a-e2d02ba43ecf admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:28.691173 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0b10e-5312-488b-8a1a-e2d02ba43ecf admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:28.696656 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41d049-9a14-498e-865a-d3677dbb1d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:28.697260 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d049-9a14-498e-865a-d3677dbb1d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.697529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d049-9a14-498e-865a-d3677dbb1d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.697830 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a41d049-9a14-498e-865a-d3677dbb1d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.698470 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41d049-9a14-498e-865a-d3677dbb1d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:28.700064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a41d049-9a14-498e-865a-d3677dbb1d89 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:28.710608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1459/2924] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:28.715087 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c01d77-a14b-48ec-9359-b495d7036902 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:28.725782 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c01d77-a14b-48ec-9359-b495d7036902 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:28.731474 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814a93b8-d117-4bd2-bb32-ec24ec57c3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:28.732108 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a93b8-d117-4bd2-bb32-ec24ec57c3cb admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.732383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a93b8-d117-4bd2-bb32-ec24ec57c3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.732628 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814a93b8-d117-4bd2-bb32-ec24ec57c3cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.733268 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a93b8-d117-4bd2-bb32-ec24ec57c3cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:28.734929 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814a93b8-d117-4bd2-bb32-ec24ec57c3cb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:28.745856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1466/2925] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:28.750493 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41d049-9a14-498e-865a-d3677dbb1d89 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:28.761758 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a41d049-9a14-498e-865a-d3677dbb1d89 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:28.766618 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34efc09-b036-4c5b-a475-2061ad8be2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:28.767280 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34efc09-b036-4c5b-a475-2061ad8be2ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.767554 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34efc09-b036-4c5b-a475-2061ad8be2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.767823 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34efc09-b036-4c5b-a475-2061ad8be2ae admin admin] PATH_INFO: `/v3/projects/f5e3f85c03234b0ca4db0d4288d2eeae/users/7fd97ca59e1748a29bc4c0813fd7de62/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.782632 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34efc09-b036-4c5b-a475-2061ad8be2ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e3f85c03234b0ca4db0d4288d2eeae, user_id=7fd97ca59e1748a29bc4c0813fd7de62, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:28.785271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34efc09-b036-4c5b-a475-2061ad8be2ae admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:28.799528 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1460/2926] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:28 2022] PUT /identity/v3/projects/f5e3f85c03234b0ca4db0d4288d2eeae/users/7fd97ca59e1748a29bc4c0813fd7de62/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:28.809567 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db2fb7-97ed-40ea-ba71-80d84975421f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:28.809879 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db2fb7-97ed-40ea-ba71-80d84975421f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:28.810181 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db2fb7-97ed-40ea-ba71-80d84975421f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:28.838301 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-07db2fb7-97ed-40ea-ba71-80d84975421f None None] MFA Rules not processed for user `7fd97ca59e1748a29bc4c0813fd7de62`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:28.839421 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-07db2fb7-97ed-40ea-ba71-80d84975421f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:28.926156 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1467/2927] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:28.932187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34efc09-b036-4c5b-a475-2061ad8be2ae admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:28.999324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34efc09-b036-4c5b-a475-2061ad8be2ae admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:29.005099 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b6b19-77c0-4685-98b1-21be1bbb9916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:29.005741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b6b19-77c0-4685-98b1-21be1bbb9916 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.006016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b6b19-77c0-4685-98b1-21be1bbb9916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.006312 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b6b19-77c0-4685-98b1-21be1bbb9916 admin admin] PATH_INFO: `/v3/projects` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.006932 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b6b19-77c0-4685-98b1-21be1bbb9916 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:29.008836 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8b6b19-77c0-4685-98b1-21be1bbb9916 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:29.038318 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1461/2928] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:02:28 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:29.042490 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db2fb7-97ed-40ea-ba71-80d84975421f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:29.053316 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db2fb7-97ed-40ea-ba71-80d84975421f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:29.065607 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:29.109489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41f44c-bb1f-4a50-91d3-6b00435d17ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:29.110058 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41f44c-bb1f-4a50-91d3-6b00435d17ed admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.110337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41f44c-bb1f-4a50-91d3-6b00435d17ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.110605 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41f44c-bb1f-4a50-91d3-6b00435d17ed admin admin] PATH_INFO: `/v3/users` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.111232 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41f44c-bb1f-4a50-91d3-6b00435d17ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:29.113168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41f44c-bb1f-4a50-91d3-6b00435d17ed admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:29.135150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1468/2929] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:02:29 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 05 15:02:29.140428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b6b19-77c0-4685-98b1-21be1bbb9916 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:29.154505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8b6b19-77c0-4685-98b1-21be1bbb9916 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:29.161327 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b0b95-4f08-48fd-bda5-699e66047dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:29.162120 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b0b95-4f08-48fd-bda5-699e66047dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.162390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b0b95-4f08-48fd-bda5-699e66047dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.162658 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85b0b95-4f08-48fd-bda5-699e66047dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.163288 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b0b95-4f08-48fd-bda5-699e66047dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:29.165514 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85b0b95-4f08-48fd-bda5-699e66047dc2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:29.178229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1462/2930] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:29.184216 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41f44c-bb1f-4a50-91d3-6b00435d17ed admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:29.197060 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41f44c-bb1f-4a50-91d3-6b00435d17ed admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:29.204319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24c633-f187-4d4b-b068-d29c1b038d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:29.204972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24c633-f187-4d4b-b068-d29c1b038d33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.205246 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24c633-f187-4d4b-b068-d29c1b038d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.205505 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24c633-f187-4d4b-b068-d29c1b038d33 admin admin] PATH_INFO: `/v3/projects/18dc9023b67c4d25bfbdd760bcfcb72a/users/596f76c826734da98a87c2aaa250835a/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.224007 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24c633-f187-4d4b-b068-d29c1b038d33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18dc9023b67c4d25bfbdd760bcfcb72a, user_id=596f76c826734da98a87c2aaa250835a, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:29.226772 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24c633-f187-4d4b-b068-d29c1b038d33 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:29.242943 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1469/2931] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:29 2022] PUT /identity/v3/projects/18dc9023b67c4d25bfbdd760bcfcb72a/users/596f76c826734da98a87c2aaa250835a/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:29.247079 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b0b95-4f08-48fd-bda5-699e66047dc2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:29.257594 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85b0b95-4f08-48fd-bda5-699e66047dc2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:29.325653 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d1d465-9b67-4806-92f7-4d269051db6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:29.326301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1d465-9b67-4806-92f7-4d269051db6f admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.326578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1d465-9b67-4806-92f7-4d269051db6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.326838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d1d465-9b67-4806-92f7-4d269051db6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.327486 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1d465-9b67-4806-92f7-4d269051db6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:29.329119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d1d465-9b67-4806-92f7-4d269051db6f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:29.339328 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1463/2932] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 15:02:29.343899 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24c633-f187-4d4b-b068-d29c1b038d33 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:29.355053 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24c633-f187-4d4b-b068-d29c1b038d33 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:29.361363 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e1159a-29e4-4e96-8bf2-c6f5a2089c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:29.362284 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1159a-29e4-4e96-8bf2-c6f5a2089c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.362671 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1159a-29e4-4e96-8bf2-c6f5a2089c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.363035 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e1159a-29e4-4e96-8bf2-c6f5a2089c12 admin admin] PATH_INFO: `/v3/projects/18dc9023b67c4d25bfbdd760bcfcb72a/users/596f76c826734da98a87c2aaa250835a/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.366128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1159a-29e4-4e96-8bf2-c6f5a2089c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18dc9023b67c4d25bfbdd760bcfcb72a, user_id=596f76c826734da98a87c2aaa250835a, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:29.370236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e1159a-29e4-4e96-8bf2-c6f5a2089c12 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:29.381969 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1470/2933] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:29 2022] PUT /identity/v3/projects/18dc9023b67c4d25bfbdd760bcfcb72a/users/596f76c826734da98a87c2aaa250835a/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:29.393209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251daa80-35bf-4861-b62b-e22ce8b53070 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.393489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251daa80-35bf-4861-b62b-e22ce8b53070 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.393798 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251daa80-35bf-4861-b62b-e22ce8b53070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.422276 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-251daa80-35bf-4861-b62b-e22ce8b53070 None None] MFA Rules not processed for user `596f76c826734da98a87c2aaa250835a`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:29.423960 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-251daa80-35bf-4861-b62b-e22ce8b53070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:29.441782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:29.502388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1464/2934] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:29.513741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b99d-f2d2-49cd-8020-e8bd521142cb None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.514018 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b99d-f2d2-49cd-8020-e8bd521142cb None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.514280 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398b99d-f2d2-49cd-8020-e8bd521142cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.556986 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7398b99d-f2d2-49cd-8020-e8bd521142cb None None] MFA Rules not processed for user `44fba9b1749b4172bc9479ca980fcf48`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:29.558236 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7398b99d-f2d2-49cd-8020-e8bd521142cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:29.612982 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1471/2935] 10.222.0.23 () {58 vars in 1060 bytes} [Wed Oct 5 15:02:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:29.619131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251daa80-35bf-4861-b62b-e22ce8b53070 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:29.629772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251daa80-35bf-4861-b62b-e22ce8b53070 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:29.634150 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c0c32-71ca-4c77-9a4d-f1d5245312d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:29.634760 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0c32-71ca-4c77-9a4d-f1d5245312d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.635035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0c32-71ca-4c77-9a4d-f1d5245312d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.635303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0c32-71ca-4c77-9a4d-f1d5245312d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.644679 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c0c32-71ca-4c77-9a4d-f1d5245312d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:29.646178 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c0c32-71ca-4c77-9a4d-f1d5245312d0 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:29.655766 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1465/2936] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:29 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:29.742096 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398b99d-f2d2-49cd-8020-e8bd521142cb None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:29.750549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398b99d-f2d2-49cd-8020-e8bd521142cb None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:29.754996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d54c364-d931-42e4-88dd-b68bb53ebbf8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:29.755599 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54c364-d931-42e4-88dd-b68bb53ebbf8 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:29.755891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54c364-d931-42e4-88dd-b68bb53ebbf8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:29.756154 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d54c364-d931-42e4-88dd-b68bb53ebbf8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:29.819679 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54c364-d931-42e4-88dd-b68bb53ebbf8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:29.821168 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d54c364-d931-42e4-88dd-b68bb53ebbf8 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:29.833297 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1472/2937] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:39.651187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=28 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:39.828856 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:40.923277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c0c32-71ca-4c77-9a4d-f1d5245312d0 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:40.987407 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c0c32-71ca-4c77-9a4d-f1d5245312d0 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:40.992494 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:40.993201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:40.993474 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:40.993788 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] PATH_INFO: `/v3/users/7fd97ca59e1748a29bc4c0813fd7de62` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:40.994967 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd97ca59e1748a29bc4c0813fd7de62)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:40.996882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:41.043926 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd97ca59e1748a29bc4c0813fd7de62'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.050824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd97ca59e1748a29bc4c0813fd7de62'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.051488 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd97ca59e1748a29bc4c0813fd7de62'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.059375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd97ca59e1748a29bc4c0813fd7de62'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.073078 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd97ca59e1748a29bc4c0813fd7de62'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.079181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1466/2938] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:02:40 2022] DELETE /identity/v3/users/7fd97ca59e1748a29bc4c0813fd7de62 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:41.083860 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d4f1d-c49a-4561-8def-4ac0800c70a3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:41.084370 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d4f1d-c49a-4561-8def-4ac0800c70a3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:41.084486 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435d4f1d-c49a-4561-8def-4ac0800c70a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:41.104321 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-435d4f1d-c49a-4561-8def-4ac0800c70a3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:41.105554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-435d4f1d-c49a-4561-8def-4ac0800c70a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:41.176303 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1473/2939] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:41.186566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:41.188175 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:41.247918 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7e4312-4b3d-4bc8-a8f3-9a29acdf0af1 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:41.251956 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6accd9d-ab45-4f1c-93f0-da64dbcb2ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:41.252575 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6accd9d-ab45-4f1c-93f0-da64dbcb2ebf service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:41.252838 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6accd9d-ab45-4f1c-93f0-da64dbcb2ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:41.253110 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6accd9d-ab45-4f1c-93f0-da64dbcb2ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:41.262657 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6accd9d-ab45-4f1c-93f0-da64dbcb2ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:41.264112 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6accd9d-ab45-4f1c-93f0-da64dbcb2ebf service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:41.274239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1467/2940] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:41.546214 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d4f1d-c49a-4561-8def-4ac0800c70a3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:41.547840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-435d4f1d-c49a-4561-8def-4ac0800c70a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:41.560462 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435d4f1d-c49a-4561-8def-4ac0800c70a3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:41.567285 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:41.567873 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:41.568123 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:41.568389 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] PATH_INFO: `/v3/projects/f5e3f85c03234b0ca4db0d4288d2eeae` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:41.570241 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e3f85c03234b0ca4db0d4288d2eeae)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:41.572343 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:41.622867 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e3f85c03234b0ca4db0d4288d2eeae', 'request_id': 'req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.628484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e3f85c03234b0ca4db0d4288d2eeae', 'request_id': 'req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.634701 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e3f85c03234b0ca4db0d4288d2eeae', 'request_id': 'req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.649946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] The token cache is being invalidate because project f5e3f85c03234b0ca4db0d4288d2eeae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:02:41.650890 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.651571 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.653880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1474/2941] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:02:41 2022] DELETE /identity/v3/projects/f5e3f85c03234b0ca4db0d4288d2eeae => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 15:02:41.663096 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6accd9d-ab45-4f1c-93f0-da64dbcb2ebf service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:41.675270 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6accd9d-ab45-4f1c-93f0-da64dbcb2ebf service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:41.693782 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:41.746144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:41.746786 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:41.747063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:41.747333 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] PATH_INFO: `/v3/users/596f76c826734da98a87c2aaa250835a` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:41.748524 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=596f76c826734da98a87c2aaa250835a)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:41.750478 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:41.800074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '596f76c826734da98a87c2aaa250835a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.807147 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '596f76c826734da98a87c2aaa250835a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.807847 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '596f76c826734da98a87c2aaa250835a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.816313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '596f76c826734da98a87c2aaa250835a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.829014 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-47e2a495-78c5-45ec-89cd-ae3a7858a5b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '596f76c826734da98a87c2aaa250835a'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:41.835452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1468/2942] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:02:41 2022] DELETE /identity/v3/users/596f76c826734da98a87c2aaa250835a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 05 15:02:42.110713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.112437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:42.127865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febf1ab-e1af-40e9-9b9b-c058e6a78f5d admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:42.188684 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:42.189231 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.189503 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.189810 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] PATH_INFO: `/v3/projects/18dc9023b67c4d25bfbdd760bcfcb72a` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.191082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18dc9023b67c4d25bfbdd760bcfcb72a)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:42.193093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:42.224893 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:42.242807 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18dc9023b67c4d25bfbdd760bcfcb72a', 'request_id': 'req-9a299030-cfb3-49cc-82ff-20200804a2a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:42.248436 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18dc9023b67c4d25bfbdd760bcfcb72a', 'request_id': 'req-9a299030-cfb3-49cc-82ff-20200804a2a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:42.254946 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18dc9023b67c4d25bfbdd760bcfcb72a', 'request_id': 'req-9a299030-cfb3-49cc-82ff-20200804a2a2'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:42.269345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] The token cache is being invalidate because project 18dc9023b67c4d25bfbdd760bcfcb72a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:02:42.269649 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:42.270293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-9a299030-cfb3-49cc-82ff-20200804a2a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:42.271809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1475/2943] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:02:42 2022] DELETE /identity/v3/projects/18dc9023b67c4d25bfbdd760bcfcb72a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:42.277055 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac002ddf-e552-439c-a76f-1e437e30004b None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.277555 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac002ddf-e552-439c-a76f-1e437e30004b None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.277707 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac002ddf-e552-439c-a76f-1e437e30004b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.299084 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-ac002ddf-e552-439c-a76f-1e437e30004b None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:42.300303 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-ac002ddf-e552-439c-a76f-1e437e30004b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:42.365130 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1469/2944] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 05 15:02:42.373372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ece913-de7d-4c08-82ee-573c53e4acc1 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.373674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ece913-de7d-4c08-82ee-573c53e4acc1 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.374179 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ece913-de7d-4c08-82ee-573c53e4acc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.395281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-34ece913-de7d-4c08-82ee-573c53e4acc1 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:42.396499 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-34ece913-de7d-4c08-82ee-573c53e4acc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:42.406390 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1476/2945] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:42.410862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac002ddf-e552-439c-a76f-1e437e30004b None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.421183 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac002ddf-e552-439c-a76f-1e437e30004b None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:42.426551 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:42.427156 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.427431 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.427700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.428348 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 15:02:42.428648 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:42.430340 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:42.442389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1470/2946] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 15:02:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 15:02:42.446800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ece913-de7d-4c08-82ee-573c53e4acc1 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.456228 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ece913-de7d-4c08-82ee-573c53e4acc1 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:42.461549 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81d637-136d-4212-9804-4c9c129d4a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:42.462153 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81d637-136d-4212-9804-4c9c129d4a13 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.462465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81d637-136d-4212-9804-4c9c129d4a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.462738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f81d637-136d-4212-9804-4c9c129d4a13 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.463718 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81d637-136d-4212-9804-4c9c129d4a13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:42.465674 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f81d637-136d-4212-9804-4c9c129d4a13 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:42.494322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1477/2947] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:02:42 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:42.498538 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.509184 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022a7576-3b5c-49a4-b604-f38a75e24114 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:42.536252 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:42.568751 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b332d32-2151-4de4-af74-d022268bd0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:42.569365 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332d32-2151-4de4-af74-d022268bd0fb admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.569666 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332d32-2151-4de4-af74-d022268bd0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.571144 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b332d32-2151-4de4-af74-d022268bd0fb admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.571888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b332d32-2151-4de4-af74-d022268bd0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:42.576181 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b332d32-2151-4de4-af74-d022268bd0fb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:42.601627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1471/2948] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:02:42 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:42.605994 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81d637-136d-4212-9804-4c9c129d4a13 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.617315 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f81d637-136d-4212-9804-4c9c129d4a13 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:42.623465 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a4ce3-361d-44c8-ac76-f02207065443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:42.624086 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a4ce3-361d-44c8-ac76-f02207065443 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.624384 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a4ce3-361d-44c8-ac76-f02207065443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.624648 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a4ce3-361d-44c8-ac76-f02207065443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.625293 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a4ce3-361d-44c8-ac76-f02207065443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:42.626948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0a4ce3-361d-44c8-ac76-f02207065443 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:42.637608 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1478/2949] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:42.641842 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b332d32-2151-4de4-af74-d022268bd0fb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.652681 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b332d32-2151-4de4-af74-d022268bd0fb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:42.658231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e817a8-cd99-424a-ae6f-263c76673532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:42.658833 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e817a8-cd99-424a-ae6f-263c76673532 admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.659106 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e817a8-cd99-424a-ae6f-263c76673532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.659375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e817a8-cd99-424a-ae6f-263c76673532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.660019 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e817a8-cd99-424a-ae6f-263c76673532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:42.661602 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e817a8-cd99-424a-ae6f-263c76673532 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:42.671817 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1472/2950] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:42.675945 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a4ce3-361d-44c8-ac76-f02207065443 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.686281 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0a4ce3-361d-44c8-ac76-f02207065443 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:42.691863 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829d0c0-c12a-4b63-a9f4-8be316ab33b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:42.692577 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829d0c0-c12a-4b63-a9f4-8be316ab33b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.692800 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829d0c0-c12a-4b63-a9f4-8be316ab33b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.693069 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829d0c0-c12a-4b63-a9f4-8be316ab33b7 admin admin] PATH_INFO: `/v3/projects/338088c325d549dbac15bb3cbab176c2/users/cba8f08592404e6aa8085314f9e239df/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.708592 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829d0c0-c12a-4b63-a9f4-8be316ab33b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338088c325d549dbac15bb3cbab176c2, user_id=cba8f08592404e6aa8085314f9e239df, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:42.711372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829d0c0-c12a-4b63-a9f4-8be316ab33b7 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:42.725676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1479/2951] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:42 2022] PUT /identity/v3/projects/338088c325d549dbac15bb3cbab176c2/users/cba8f08592404e6aa8085314f9e239df/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:42.737285 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34384d05-05a4-44ca-a84e-787f4e5331ce None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.737569 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34384d05-05a4-44ca-a84e-787f4e5331ce None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.737908 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34384d05-05a4-44ca-a84e-787f4e5331ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.767397 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-34384d05-05a4-44ca-a84e-787f4e5331ce None None] MFA Rules not processed for user `cba8f08592404e6aa8085314f9e239df`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:42.768615 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-34384d05-05a4-44ca-a84e-787f4e5331ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:42.833938 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1473/2952] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:42.839147 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829d0c0-c12a-4b63-a9f4-8be316ab33b7 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.903758 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829d0c0-c12a-4b63-a9f4-8be316ab33b7 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:42.908896 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aface84f-9c99-4ff5-bf69-bf73697dcaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:42.909500 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aface84f-9c99-4ff5-bf69-bf73697dcaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:42.909809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aface84f-9c99-4ff5-bf69-bf73697dcaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:42.910084 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aface84f-9c99-4ff5-bf69-bf73697dcaa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:42.910716 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aface84f-9c99-4ff5-bf69-bf73697dcaa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:42.912641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aface84f-9c99-4ff5-bf69-bf73697dcaa8 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:42.941033 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1480/2953] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:02:42 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:42.945719 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34384d05-05a4-44ca-a84e-787f4e5331ce None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:42.956616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34384d05-05a4-44ca-a84e-787f4e5331ce None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:43.014265 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025b3d0-7f7d-4708-bec5-4920e151e861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:43.014875 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025b3d0-7f7d-4708-bec5-4920e151e861 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:43.015155 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025b3d0-7f7d-4708-bec5-4920e151e861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:43.015425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025b3d0-7f7d-4708-bec5-4920e151e861 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:43.016061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025b3d0-7f7d-4708-bec5-4920e151e861 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:43.018013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025b3d0-7f7d-4708-bec5-4920e151e861 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:43.040738 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1474/2954] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:02:42 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:43.045093 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aface84f-9c99-4ff5-bf69-bf73697dcaa8 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:43.055617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aface84f-9c99-4ff5-bf69-bf73697dcaa8 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:43.060437 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43577a0f-5f7f-4017-8b8b-bbd719c72e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:43.061041 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43577a0f-5f7f-4017-8b8b-bbd719c72e5c admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:43.061319 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43577a0f-5f7f-4017-8b8b-bbd719c72e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:43.061585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43577a0f-5f7f-4017-8b8b-bbd719c72e5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:43.062253 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43577a0f-5f7f-4017-8b8b-bbd719c72e5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:43.063865 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43577a0f-5f7f-4017-8b8b-bbd719c72e5c admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:43.074546 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1481/2955] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:43.078891 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025b3d0-7f7d-4708-bec5-4920e151e861 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:43.089371 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025b3d0-7f7d-4708-bec5-4920e151e861 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:43.094209 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64f255-15d0-4479-ad46-aceb1901e414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:43.094861 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64f255-15d0-4479-ad46-aceb1901e414 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:43.095142 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64f255-15d0-4479-ad46-aceb1901e414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:43.095414 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e64f255-15d0-4479-ad46-aceb1901e414 admin admin] PATH_INFO: `/v3/projects/c02394ea8dee42718b3152f6a18b9a55/users/6122e611162f4afaafee3bd48ce2c991/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:43.110360 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64f255-15d0-4479-ad46-aceb1901e414 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02394ea8dee42718b3152f6a18b9a55, user_id=6122e611162f4afaafee3bd48ce2c991, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:43.113040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e64f255-15d0-4479-ad46-aceb1901e414 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:43.127199 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1475/2956] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:43 2022] PUT /identity/v3/projects/c02394ea8dee42718b3152f6a18b9a55/users/6122e611162f4afaafee3bd48ce2c991/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:43.131385 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43577a0f-5f7f-4017-8b8b-bbd719c72e5c admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:43.141741 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43577a0f-5f7f-4017-8b8b-bbd719c72e5c admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:43.162989 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:43.200761 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c9d8e-242a-4d2d-ae56-f6913911ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:43.201325 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c9d8e-242a-4d2d-ae56-f6913911ddf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:43.201594 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c9d8e-242a-4d2d-ae56-f6913911ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:43.201891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c9d8e-242a-4d2d-ae56-f6913911ddf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:43.202525 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c9d8e-242a-4d2d-ae56-f6913911ddf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:43.204160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c9d8e-242a-4d2d-ae56-f6913911ddf5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:43.214068 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1482/2957] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 15:02:43.218532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64f255-15d0-4479-ad46-aceb1901e414 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:43.224063 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:43.229532 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e64f255-15d0-4479-ad46-aceb1901e414 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:43.235275 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:43.235923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:43.236191 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:43.236455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] PATH_INFO: `/v3/projects/c02394ea8dee42718b3152f6a18b9a55/users/6122e611162f4afaafee3bd48ce2c991/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:43.238481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c02394ea8dee42718b3152f6a18b9a55, user_id=6122e611162f4afaafee3bd48ce2c991, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:43.241224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:43.252359 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1476/2958] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:43 2022] PUT /identity/v3/projects/c02394ea8dee42718b3152f6a18b9a55/users/6122e611162f4afaafee3bd48ce2c991/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:43.263118 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9de89b-c55c-4a35-b736-ad3927b90dc9 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:43.263397 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9de89b-c55c-4a35-b736-ad3927b90dc9 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:43.263665 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9de89b-c55c-4a35-b736-ad3927b90dc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:43.291626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ba9de89b-c55c-4a35-b736-ad3927b90dc9 None None] MFA Rules not processed for user `6122e611162f4afaafee3bd48ce2c991`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:43.292840 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ba9de89b-c55c-4a35-b736-ad3927b90dc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:43.362010 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1483/2959] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:43.374563 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:43.376447 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:43.435374 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8ae3b-842d-4bec-91e2-9a44bb38b4b2 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:43.439406 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed7ec4-7ddf-415e-9015-8eebae0517a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:43.440009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7ec4-7ddf-415e-9015-8eebae0517a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:43.440306 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7ec4-7ddf-415e-9015-8eebae0517a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:43.440549 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed7ec4-7ddf-415e-9015-8eebae0517a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:43.450300 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed7ec4-7ddf-415e-9015-8eebae0517a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:43.451824 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed7ec4-7ddf-415e-9015-8eebae0517a9 service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:43.462102 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1477/2960] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:43.542057 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9de89b-c55c-4a35-b736-ad3927b90dc9 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:43.552593 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9de89b-c55c-4a35-b736-ad3927b90dc9 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:43.556997 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc823fd-009e-42d6-841c-a6a6115fe33f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:43.557598 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc823fd-009e-42d6-841c-a6a6115fe33f service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:43.557898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc823fd-009e-42d6-841c-a6a6115fe33f service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:43.558166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc823fd-009e-42d6-841c-a6a6115fe33f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:43.616537 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc823fd-009e-42d6-841c-a6a6115fe33f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:43.617993 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc823fd-009e-42d6-841c-a6a6115fe33f service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:43.627378 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1484/2961] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:46.391772 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed7ec4-7ddf-415e-9015-8eebae0517a9 service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:46.459332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed7ec4-7ddf-415e-9015-8eebae0517a9 service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:46.464603 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:46.465229 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:46.465502 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:46.465792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] PATH_INFO: `/v3/users/cba8f08592404e6aa8085314f9e239df` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:46.466935 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cba8f08592404e6aa8085314f9e239df)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:46.468857 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:46.522785 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cba8f08592404e6aa8085314f9e239df'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:46.529548 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cba8f08592404e6aa8085314f9e239df'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:46.530298 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cba8f08592404e6aa8085314f9e239df'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:46.539846 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cba8f08592404e6aa8085314f9e239df'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:46.552877 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cba8f08592404e6aa8085314f9e239df'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:46.559044 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1478/2962] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:02:46 2022] DELETE /identity/v3/users/cba8f08592404e6aa8085314f9e239df => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:46.563745 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227572-072f-4cbc-9656-73f9b622827f None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:46.564222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227572-072f-4cbc-9656-73f9b622827f None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:46.564345 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f227572-072f-4cbc-9656-73f9b622827f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:46.584221 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-5f227572-072f-4cbc-9656-73f9b622827f None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:46.585450 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5f227572-072f-4cbc-9656-73f9b622827f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:46.635996 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=100 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:46.667202 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1485/2963] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:46.677828 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:46.679401 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:46.693148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:46.738308 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac71ca-3ff9-4fb2-9dcc-079cd7926506 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:46.742390 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fea63-33a9-482b-be68-727b90bb66e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:46.743001 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fea63-33a9-482b-be68-727b90bb66e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:46.743271 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fea63-33a9-482b-be68-727b90bb66e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:46.743529 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7fea63-33a9-482b-be68-727b90bb66e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:46.752739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fea63-33a9-482b-be68-727b90bb66e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:46.754224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7fea63-33a9-482b-be68-727b90bb66e6 service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:46.763720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1479/2964] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:47.048213 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f227572-072f-4cbc-9656-73f9b622827f None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:47.050143 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-5f227572-072f-4cbc-9656-73f9b622827f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:47.062799 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f227572-072f-4cbc-9656-73f9b622827f None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:47.068188 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:47.068968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:47.069382 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:47.069668 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] PATH_INFO: `/v3/projects/338088c325d549dbac15bb3cbab176c2` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:47.070880 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=338088c325d549dbac15bb3cbab176c2)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:47.072798 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:47.122180 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338088c325d549dbac15bb3cbab176c2', 'request_id': 'req-2c247bde-59d3-4983-9ed6-a6f7188a03ea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.127586 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338088c325d549dbac15bb3cbab176c2', 'request_id': 'req-2c247bde-59d3-4983-9ed6-a6f7188a03ea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.133934 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338088c325d549dbac15bb3cbab176c2', 'request_id': 'req-2c247bde-59d3-4983-9ed6-a6f7188a03ea'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.150184 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] The token cache is being invalidate because project 338088c325d549dbac15bb3cbab176c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:02:47.150464 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.151166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.152675 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1486/2965] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:02:47 2022] DELETE /identity/v3/projects/338088c325d549dbac15bb3cbab176c2 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:47.158425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fea63-33a9-482b-be68-727b90bb66e6 service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:47.168299 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7fea63-33a9-482b-be68-727b90bb66e6 service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:47.226284 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:47.226944 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:47.227215 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:47.227475 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] PATH_INFO: `/v3/users/6122e611162f4afaafee3bd48ce2c991` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:47.228627 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6122e611162f4afaafee3bd48ce2c991)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:47.230582 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:47.288255 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6122e611162f4afaafee3bd48ce2c991'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.295750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6122e611162f4afaafee3bd48ce2c991'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.296389 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6122e611162f4afaafee3bd48ce2c991'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.305560 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6122e611162f4afaafee3bd48ce2c991'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.321887 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-dabded9a-9bca-47a7-8526-2228e7f56395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6122e611162f4afaafee3bd48ce2c991'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.327978 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1480/2966] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:02:47 2022] DELETE /identity/v3/users/6122e611162f4afaafee3bd48ce2c991 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:47.606003 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:47.607547 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:47.621585 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c247bde-59d3-4983-9ed6-a6f7188a03ea admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:47.685983 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:47.686621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:47.686898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:47.687166 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] PATH_INFO: `/v3/projects/c02394ea8dee42718b3152f6a18b9a55` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:47.688295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c02394ea8dee42718b3152f6a18b9a55)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:47.690238 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:47.739532 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c02394ea8dee42718b3152f6a18b9a55', 'request_id': 'req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.745222 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c02394ea8dee42718b3152f6a18b9a55', 'request_id': 'req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.751968 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c02394ea8dee42718b3152f6a18b9a55', 'request_id': 'req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.761373 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:47.766730 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] The token cache is being invalidate because project c02394ea8dee42718b3152f6a18b9a55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:02:47.767026 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.767878 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-7b3367cd-f0e7-45f4-b57a-c6099bbd758a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:47.770092 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1487/2967] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:02:47 2022] DELETE /identity/v3/projects/c02394ea8dee42718b3152f6a18b9a55 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:47.776962 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1134-d4fc-472a-9f13-2406ab936021 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:47.777484 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1134-d4fc-472a-9f13-2406ab936021 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:47.777596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070a1134-d4fc-472a-9f13-2406ab936021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:47.797862 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-070a1134-d4fc-472a-9f13-2406ab936021 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:47.799125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-070a1134-d4fc-472a-9f13-2406ab936021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:47.829674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:47.861721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1481/2968] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:47.869617 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d917-2613-44fc-b59e-ba99da14cfe2 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:47.869931 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d917-2613-44fc-b59e-ba99da14cfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:47.870207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad24d917-2613-44fc-b59e-ba99da14cfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:47.889824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-ad24d917-2613-44fc-b59e-ba99da14cfe2 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:47.891029 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-ad24d917-2613-44fc-b59e-ba99da14cfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:47.899797 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1488/2969] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:47.904035 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1134-d4fc-472a-9f13-2406ab936021 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:47.914074 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070a1134-d4fc-472a-9f13-2406ab936021 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:47.919341 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:47.919972 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:47.920243 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:47.920505 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:47.921162 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 15:02:47.921439 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:47.923061 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:47.934866 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1482/2970] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 15:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 15:02:47.939234 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24d917-2613-44fc-b59e-ba99da14cfe2 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:47.948489 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad24d917-2613-44fc-b59e-ba99da14cfe2 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:47.953589 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a21af-3ada-4c4e-a2d8-fd49c3d1a423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:47.954288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a21af-3ada-4c4e-a2d8-fd49c3d1a423 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:47.954562 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a21af-3ada-4c4e-a2d8-fd49c3d1a423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:47.954824 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a21af-3ada-4c4e-a2d8-fd49c3d1a423 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:47.955443 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a21af-3ada-4c4e-a2d8-fd49c3d1a423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:47.957342 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a21af-3ada-4c4e-a2d8-fd49c3d1a423 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:47.988829 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1489/2971] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:02:47 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:47.993578 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.008997 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1dd9fc-d730-4d05-bfcf-f6bb549e9b1d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.073129 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1523-16b3-4716-a1d3-7b355cb99d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.073761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1523-16b3-4716-a1d3-7b355cb99d2f admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.074051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1523-16b3-4716-a1d3-7b355cb99d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.074318 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a1523-16b3-4716-a1d3-7b355cb99d2f admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.074953 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a1523-16b3-4716-a1d3-7b355cb99d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.076871 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107a1523-16b3-4716-a1d3-7b355cb99d2f admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.099326 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1483/2972] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:02:47 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:48.103492 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a21af-3ada-4c4e-a2d8-fd49c3d1a423 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.114079 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a21af-3ada-4c4e-a2d8-fd49c3d1a423 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.119289 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42713bb-7904-4ce1-9c31-6f7bf950be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.119857 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42713bb-7904-4ce1-9c31-6f7bf950be50 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.120128 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42713bb-7904-4ce1-9c31-6f7bf950be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.120444 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42713bb-7904-4ce1-9c31-6f7bf950be50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.121077 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42713bb-7904-4ce1-9c31-6f7bf950be50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.122696 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42713bb-7904-4ce1-9c31-6f7bf950be50 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.132650 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1490/2973] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:48.136822 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1523-16b3-4716-a1d3-7b355cb99d2f admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.147567 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107a1523-16b3-4716-a1d3-7b355cb99d2f admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.153028 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26db6bc-7921-4ebd-8ef1-2b745ed8562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.153739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db6bc-7921-4ebd-8ef1-2b745ed8562a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.153965 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db6bc-7921-4ebd-8ef1-2b745ed8562a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.154239 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26db6bc-7921-4ebd-8ef1-2b745ed8562a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.154888 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26db6bc-7921-4ebd-8ef1-2b745ed8562a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.156481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26db6bc-7921-4ebd-8ef1-2b745ed8562a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.167344 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1484/2974] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:48.171670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42713bb-7904-4ce1-9c31-6f7bf950be50 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.181939 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42713bb-7904-4ce1-9c31-6f7bf950be50 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.187502 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29602-7280-4803-b1bf-ffd75ee8cab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.188349 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29602-7280-4803-b1bf-ffd75ee8cab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.188623 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29602-7280-4803-b1bf-ffd75ee8cab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.188887 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad29602-7280-4803-b1bf-ffd75ee8cab5 admin admin] PATH_INFO: `/v3/projects/69016078c72a4c98947965b3936d63a7/users/ee4972e8f87c4eae858aa11fed783d18/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.203814 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad29602-7280-4803-b1bf-ffd75ee8cab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69016078c72a4c98947965b3936d63a7, user_id=ee4972e8f87c4eae858aa11fed783d18, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.206522 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad29602-7280-4803-b1bf-ffd75ee8cab5 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.222192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1491/2975] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:48 2022] PUT /identity/v3/projects/69016078c72a4c98947965b3936d63a7/users/ee4972e8f87c4eae858aa11fed783d18/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:48.233425 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926cbcd-453d-4fe7-a847-482aa144d33d None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.233741 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926cbcd-453d-4fe7-a847-482aa144d33d None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.234013 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c926cbcd-453d-4fe7-a847-482aa144d33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.262176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-c926cbcd-453d-4fe7-a847-482aa144d33d None None] MFA Rules not processed for user `ee4972e8f87c4eae858aa11fed783d18`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:48.263363 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-c926cbcd-453d-4fe7-a847-482aa144d33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:48.345064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1485/2976] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:48.350046 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29602-7280-4803-b1bf-ffd75ee8cab5 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.425295 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad29602-7280-4803-b1bf-ffd75ee8cab5 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.430555 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696145f-4c07-456c-a617-b4f03e0c58fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.431167 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696145f-4c07-456c-a617-b4f03e0c58fb admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.431449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696145f-4c07-456c-a617-b4f03e0c58fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.431713 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696145f-4c07-456c-a617-b4f03e0c58fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.432337 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696145f-4c07-456c-a617-b4f03e0c58fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.434396 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696145f-4c07-456c-a617-b4f03e0c58fb admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.469647 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1492/2977] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:02:48 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:48.474048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926cbcd-453d-4fe7-a847-482aa144d33d None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.484805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c926cbcd-453d-4fe7-a847-482aa144d33d None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.511247 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:48.542695 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a468b7-e105-41f7-bc78-18646d08cf89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.543307 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a468b7-e105-41f7-bc78-18646d08cf89 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.543583 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a468b7-e105-41f7-bc78-18646d08cf89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.543848 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a468b7-e105-41f7-bc78-18646d08cf89 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.544485 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a468b7-e105-41f7-bc78-18646d08cf89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.546445 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a468b7-e105-41f7-bc78-18646d08cf89 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.576650 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1486/2978] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:02:48 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:02:48.581426 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696145f-4c07-456c-a617-b4f03e0c58fb admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.597604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696145f-4c07-456c-a617-b4f03e0c58fb admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.607535 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7501789-a581-424a-99f0-813144fba8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.608523 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7501789-a581-424a-99f0-813144fba8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.608844 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7501789-a581-424a-99f0-813144fba8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.609155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7501789-a581-424a-99f0-813144fba8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.609872 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7501789-a581-424a-99f0-813144fba8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.611509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7501789-a581-424a-99f0-813144fba8a4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.626789 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1493/2979] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:48.633853 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a468b7-e105-41f7-bc78-18646d08cf89 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.649759 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a468b7-e105-41f7-bc78-18646d08cf89 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.655369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbf740-fcba-4df2-99c6-32d6aa24dc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.656020 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbf740-fcba-4df2-99c6-32d6aa24dc7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.656295 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbf740-fcba-4df2-99c6-32d6aa24dc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.656566 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dbf740-fcba-4df2-99c6-32d6aa24dc7d admin admin] PATH_INFO: `/v3/projects/9ffee4e5902e456f8e7fbcb10d7f9c49/users/b2eac06515a04d31859ad00010b3a0db/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.671863 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbf740-fcba-4df2-99c6-32d6aa24dc7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffee4e5902e456f8e7fbcb10d7f9c49, user_id=b2eac06515a04d31859ad00010b3a0db, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.674596 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dbf740-fcba-4df2-99c6-32d6aa24dc7d admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.689325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1487/2980] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:48 2022] PUT /identity/v3/projects/9ffee4e5902e456f8e7fbcb10d7f9c49/users/b2eac06515a04d31859ad00010b3a0db/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:48.693261 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7501789-a581-424a-99f0-813144fba8a4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.703473 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7501789-a581-424a-99f0-813144fba8a4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.761513 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6b90e8-baa1-45a6-bfdc-b72f2440841e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.762171 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b90e8-baa1-45a6-bfdc-b72f2440841e admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.762453 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b90e8-baa1-45a6-bfdc-b72f2440841e admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.762744 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6b90e8-baa1-45a6-bfdc-b72f2440841e admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.763372 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b90e8-baa1-45a6-bfdc-b72f2440841e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.764972 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6b90e8-baa1-45a6-bfdc-b72f2440841e admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.774935 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1494/2981] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:02:48.779210 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbf740-fcba-4df2-99c6-32d6aa24dc7d admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.789903 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dbf740-fcba-4df2-99c6-32d6aa24dc7d admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:48.794674 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:48.795332 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.795608 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.795878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] PATH_INFO: `/v3/projects/9ffee4e5902e456f8e7fbcb10d7f9c49/users/b2eac06515a04d31859ad00010b3a0db/roles/b9327f2ca9284c5ea652cbce8217b47d` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.797832 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ffee4e5902e456f8e7fbcb10d7f9c49, user_id=b2eac06515a04d31859ad00010b3a0db, role_id=b9327f2ca9284c5ea652cbce8217b47d)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:48.800469 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:48.810576 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1488/2982] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:02:48 2022] PUT /identity/v3/projects/9ffee4e5902e456f8e7fbcb10d7f9c49/users/b2eac06515a04d31859ad00010b3a0db/roles/b9327f2ca9284c5ea652cbce8217b47d => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:48.821074 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2da0e-9c62-405b-bb73-771b40a6b440 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:48.821356 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2da0e-9c62-405b-bb73-771b40a6b440 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:48.821622 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2da0e-9c62-405b-bb73-771b40a6b440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:48.830264 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:48.852011 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-1ab2da0e-9c62-405b-bb73-771b40a6b440 None None] MFA Rules not processed for user `b2eac06515a04d31859ad00010b3a0db`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:48.853248 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1ab2da0e-9c62-405b-bb73-771b40a6b440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:48.924012 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1495/2983] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:48.934951 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:48.936835 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:48.999313 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd3328-a55b-4b71-805d-80be11a2f2cb admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:49.003244 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6476211-8561-4ce1-be0f-a80e1fd64bdc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:49.003841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6476211-8561-4ce1-be0f-a80e1fd64bdc service cinder] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:49.004111 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6476211-8561-4ce1-be0f-a80e1fd64bdc service cinder] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:49.004372 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6476211-8561-4ce1-be0f-a80e1fd64bdc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:49.064428 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6476211-8561-4ce1-be0f-a80e1fd64bdc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:49.065889 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6476211-8561-4ce1-be0f-a80e1fd64bdc service cinder] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:49.074841 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1489/2984] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:53.394692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2da0e-9c62-405b-bb73-771b40a6b440 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:53.396276 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-1ab2da0e-9c62-405b-bb73-771b40a6b440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:53.409375 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2da0e-9c62-405b-bb73-771b40a6b440 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:53.413376 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61dec5-0ffe-44c7-93b1-4db8f8adecfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:53.414013 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61dec5-0ffe-44c7-93b1-4db8f8adecfe service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:53.414287 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61dec5-0ffe-44c7-93b1-4db8f8adecfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:53.414554 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61dec5-0ffe-44c7-93b1-4db8f8adecfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:53.423944 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61dec5-0ffe-44c7-93b1-4db8f8adecfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:53.425362 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61dec5-0ffe-44c7-93b1-4db8f8adecfe service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:53.435022 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1496/2985] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:53 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:58.598204 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6476211-8561-4ce1-be0f-a80e1fd64bdc service cinder] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:58.648511 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:58.675048 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6476211-8561-4ce1-be0f-a80e1fd64bdc service cinder] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:58.679805 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:58.680433 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:58.680704 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:58.680973 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] PATH_INFO: `/v3/users/ee4972e8f87c4eae858aa11fed783d18` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:58.682138 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee4972e8f87c4eae858aa11fed783d18)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:58.684043 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:58.733131 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee4972e8f87c4eae858aa11fed783d18'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:58.740400 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee4972e8f87c4eae858aa11fed783d18'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:58.741157 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee4972e8f87c4eae858aa11fed783d18'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:58.752496 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee4972e8f87c4eae858aa11fed783d18'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:58.764616 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee4972e8f87c4eae858aa11fed783d18'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:58.771750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1490/2986] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:02:58 2022] DELETE /identity/v3/users/ee4972e8f87c4eae858aa11fed783d18 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:58.776483 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc2a9-49d3-4148-a2a5-aa56e2cd99d3 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:58.777027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc2a9-49d3-4148-a2a5-aa56e2cd99d3 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:58.777130 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684bc2a9-49d3-4148-a2a5-aa56e2cd99d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:58.796517 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-684bc2a9-49d3-4148-a2a5-aa56e2cd99d3 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:58.797792 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-684bc2a9-49d3-4148-a2a5-aa56e2cd99d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:58.857984 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1497/2987] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:02:58.868212 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:58.869750 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:58.928201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3e52b9-1d40-4bb6-b0e9-f8cf1a7b8aef admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:58.932309 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce16a8c-7cfe-4779-90f9-5053d974f8de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:58.932921 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce16a8c-7cfe-4779-90f9-5053d974f8de service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:58.933201 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce16a8c-7cfe-4779-90f9-5053d974f8de service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:58.933461 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce16a8c-7cfe-4779-90f9-5053d974f8de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:58.943064 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce16a8c-7cfe-4779-90f9-5053d974f8de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:58.944503 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce16a8c-7cfe-4779-90f9-5053d974f8de service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:58.954086 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1491/2988] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:02:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:02:59.227966 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc2a9-49d3-4148-a2a5-aa56e2cd99d3 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:59.229456 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-684bc2a9-49d3-4148-a2a5-aa56e2cd99d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:59.242941 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684bc2a9-49d3-4148-a2a5-aa56e2cd99d3 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:59.248288 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:59.248940 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:59.249192 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:59.249458 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] PATH_INFO: `/v3/projects/69016078c72a4c98947965b3936d63a7` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:59.250738 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69016078c72a4c98947965b3936d63a7)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:59.252660 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:59.300870 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69016078c72a4c98947965b3936d63a7', 'request_id': 'req-0dbcf60c-610d-417b-b19a-93e9da7d38f4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.306845 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69016078c72a4c98947965b3936d63a7', 'request_id': 'req-0dbcf60c-610d-417b-b19a-93e9da7d38f4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.313081 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69016078c72a4c98947965b3936d63a7', 'request_id': 'req-0dbcf60c-610d-417b-b19a-93e9da7d38f4'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.323971 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:59.328449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] The token cache is being invalidate because project 69016078c72a4c98947965b3936d63a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:02:59.328776 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.329544 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.331795 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1498/2989] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:02:59 2022] DELETE /identity/v3/projects/69016078c72a4c98947965b3936d63a7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:59.336052 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce16a8c-7cfe-4779-90f9-5053d974f8de service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:59.346452 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce16a8c-7cfe-4779-90f9-5053d974f8de service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:59.398625 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:02:59.405455 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:59.406188 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:59.406472 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:59.406730 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] PATH_INFO: `/v3/users/b2eac06515a04d31859ad00010b3a0db` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:59.407969 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2eac06515a04d31859ad00010b3a0db)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:59.410231 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:59.468182 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2eac06515a04d31859ad00010b3a0db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.476141 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2eac06515a04d31859ad00010b3a0db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.476923 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2eac06515a04d31859ad00010b3a0db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.486501 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2eac06515a04d31859ad00010b3a0db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.498369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-1016248c-7fbe-460f-a7d9-33e8b772dcef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2eac06515a04d31859ad00010b3a0db'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.504456 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1492/2990] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:02:59 2022] DELETE /identity/v3/users/b2eac06515a04d31859ad00010b3a0db => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:59.792621 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:02:59.794067 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:02:59.807614 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbcf60c-610d-417b-b19a-93e9da7d38f4 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:02:59.865383 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:02:59.865963 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:59.866207 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:59.866501 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] PATH_INFO: `/v3/projects/9ffee4e5902e456f8e7fbcb10d7f9c49` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:59.867604 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ffee4e5902e456f8e7fbcb10d7f9c49)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:02:59.869509 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:02:59.917269 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ffee4e5902e456f8e7fbcb10d7f9c49', 'request_id': 'req-8f710559-c8a1-4b08-83f5-a465e62a3770'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.923062 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ffee4e5902e456f8e7fbcb10d7f9c49', 'request_id': 'req-8f710559-c8a1-4b08-83f5-a465e62a3770'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.929135 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ffee4e5902e456f8e7fbcb10d7f9c49', 'request_id': 'req-8f710559-c8a1-4b08-83f5-a465e62a3770'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.943685 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] The token cache is being invalidate because project 9ffee4e5902e456f8e7fbcb10d7f9c49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:02:59.943961 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.944590 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-8f710559-c8a1-4b08-83f5-a465e62a3770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:02:59.946155 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1499/2991] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:02:59 2022] DELETE /identity/v3/projects/9ffee4e5902e456f8e7fbcb10d7f9c49 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:02:59.952187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db10497-9628-4806-9056-7c966b7b77df None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:02:59.952670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db10497-9628-4806-9056-7c966b7b77df None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:02:59.952792 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db10497-9628-4806-9056-7c966b7b77df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:02:59.974380 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-7db10497-9628-4806-9056-7c966b7b77df None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:02:59.975593 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-7db10497-9628-4806-9056-7c966b7b77df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:03:00.035149 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1493/2992] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:02:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:03:00.047265 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db374fe-7f49-4adc-a392-bc04b7e79af7 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.047518 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db374fe-7f49-4adc-a392-bc04b7e79af7 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.047920 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db374fe-7f49-4adc-a392-bc04b7e79af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.076424 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-7db374fe-7f49-4adc-a392-bc04b7e79af7 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:03:00.077692 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-7db374fe-7f49-4adc-a392-bc04b7e79af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:03:00.087568 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1500/2993] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:03:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:03:00.094036 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db10497-9628-4806-9056-7c966b7b77df None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:00.104964 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db10497-9628-4806-9056-7c966b7b77df None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:00.111277 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:00.112148 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.112481 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.112814 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] PATH_INFO: `/v3/domains` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.113813 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81571) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 05 15:03:00.114214 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:00.117009 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:00.129910 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1494/2994] 10.222.0.23 () {58 vars in 1153 bytes} [Wed Oct 5 15:03:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 05 15:03:00.134274 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db374fe-7f49-4adc-a392-bc04b7e79af7 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:00.143831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db374fe-7f49-4adc-a392-bc04b7e79af7 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:00.149830 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fef7f0-4f64-4c63-8b75-6614627f8522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:00.150435 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fef7f0-4f64-4c63-8b75-6614627f8522 admin admin] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.150711 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fef7f0-4f64-4c63-8b75-6614627f8522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.151005 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fef7f0-4f64-4c63-8b75-6614627f8522 admin admin] PATH_INFO: `/v3/projects` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.151644 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fef7f0-4f64-4c63-8b75-6614627f8522 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:00.153575 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fef7f0-4f64-4c63-8b75-6614627f8522 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:00.184259 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1501/2995] 10.222.0.23 () {60 vars in 1153 bytes} [Wed Oct 5 15:03:00 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:03:00.188604 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:00.198740 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123aafd4-76e1-43b6-8273-09daec54f95c admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:00.258301 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6cc89-ec5c-4665-bf78-5da93c01a079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:00.258906 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6cc89-ec5c-4665-bf78-5da93c01a079 admin admin] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.259207 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6cc89-ec5c-4665-bf78-5da93c01a079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.259489 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6cc89-ec5c-4665-bf78-5da93c01a079 admin admin] PATH_INFO: `/v3/users` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.260119 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6cc89-ec5c-4665-bf78-5da93c01a079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:00.262082 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6cc89-ec5c-4665-bf78-5da93c01a079 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:00.272654 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:03:00.287336 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1495/2996] 10.222.0.23 () {60 vars in 1144 bytes} [Wed Oct 5 15:03:00 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 05 15:03:00.291449 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fef7f0-4f64-4c63-8b75-6614627f8522 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:00.302322 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fef7f0-4f64-4c63-8b75-6614627f8522 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:00.308131 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb3030-5b7c-41ad-b90c-a7c14ccfe513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:00.308733 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb3030-5b7c-41ad-b90c-a7c14ccfe513 admin admin] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.309001 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb3030-5b7c-41ad-b90c-a7c14ccfe513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.309247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb3030-5b7c-41ad-b90c-a7c14ccfe513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.309921 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb3030-5b7c-41ad-b90c-a7c14ccfe513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:00.311519 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bb3030-5b7c-41ad-b90c-a7c14ccfe513 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:00.322160 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1502/2997] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 05 15:03:00.326325 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6cc89-ec5c-4665-bf78-5da93c01a079 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:00.337928 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6cc89-ec5c-4665-bf78-5da93c01a079 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:00.345495 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b665704-68cb-4359-9544-bc1b1727519a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:00.346176 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b665704-68cb-4359-9544-bc1b1727519a admin admin] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.346454 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b665704-68cb-4359-9544-bc1b1727519a admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.346718 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b665704-68cb-4359-9544-bc1b1727519a admin admin] PATH_INFO: `/v3/roles` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.347375 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b665704-68cb-4359-9544-bc1b1727519a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:00.348987 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b665704-68cb-4359-9544-bc1b1727519a admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:00.361057 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1496/2998] 10.222.0.23 () {58 vars in 1122 bytes} [Wed Oct 5 15:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 05 15:03:00.365831 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb3030-5b7c-41ad-b90c-a7c14ccfe513 admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:00.376891 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb3030-5b7c-41ad-b90c-a7c14ccfe513 admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:00.382484 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:00.383150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] REQUEST_METHOD: `PUT` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.383425 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.383676 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] PATH_INFO: `/v3/projects/7a72dc2bf186400384b99db491eef760/users/225173f51c3b46f7ae9891c71b1d461c/roles/7b4a508ded224dfc812c05d174d41511` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.400150 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a72dc2bf186400384b99db491eef760, user_id=225173f51c3b46f7ae9891c71b1d461c, role_id=7b4a508ded224dfc812c05d174d41511)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:00.402875 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:00.418124 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1503/2999] 10.222.0.23 () {60 vars in 1483 bytes} [Wed Oct 5 15:03:00 2022] PUT /identity/v3/projects/7a72dc2bf186400384b99db491eef760/users/225173f51c3b46f7ae9891c71b1d461c/roles/7b4a508ded224dfc812c05d174d41511 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:03:00.429429 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196087-f0eb-46d4-9f9d-0f284dbe58b9 None None] REQUEST_METHOD: `POST` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.429736 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196087-f0eb-46d4-9f9d-0f284dbe58b9 None None] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.430007 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3196087-f0eb-46d4-9f9d-0f284dbe58b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.459040 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.auth.core [None req-f3196087-f0eb-46d4-9f9d-0f284dbe58b9 None None] MFA Rules not processed for user `225173f51c3b46f7ae9891c71b1d461c`. Rule list: `[]` (Enabled: `True`). {{(pid=81571) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:03:00.460235 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-f3196087-f0eb-46d4-9f9d-0f284dbe58b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:03:00.530267 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1497/3000] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:03:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:03:00.542314 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:00.544082 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:03:00.575225 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:03:00.602802 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71212637-f242-49cd-9120-947fc3f7c68f admin admin] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:00.606809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f3648-c251-44be-b4de-64ea5d48d494 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44fba9b1749b4172bc9479ca980fcf48', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:00.607415 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3648-c251-44be-b4de-64ea5d48d494 service cinder] REQUEST_METHOD: `GET` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:00.607670 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3648-c251-44be-b4de-64ea5d48d494 service cinder] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:00.607988 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f3648-c251-44be-b4de-64ea5d48d494 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:00.616784 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f3648-c251-44be-b4de-64ea5d48d494 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:00.618255 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f3648-c251-44be-b4de-64ea5d48d494 service cinder] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:00.627597 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1504/3001] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:03:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:03:03.236174 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3196087-f0eb-46d4-9f9d-0f284dbe58b9 None None] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:03.308090 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3196087-f0eb-46d4-9f9d-0f284dbe58b9 None None] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:03.314095 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:03.314914 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:03.315224 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:03.315556 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] PATH_INFO: `/v3/users/225173f51c3b46f7ae9891c71b1d461c` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:03.316800 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=225173f51c3b46f7ae9891c71b1d461c)` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:03.318780 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:03.374114 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '225173f51c3b46f7ae9891c71b1d461c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.381016 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '225173f51c3b46f7ae9891c71b1d461c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.381696 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '225173f51c3b46f7ae9891c71b1d461c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.390324 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '225173f51c3b46f7ae9891c71b1d461c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.398882 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:INSERT=2,keystone:SELECT=89,keystone:DELETE=9 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:03:03.404187 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.notifications [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '225173f51c3b46f7ae9891c71b1d461c'} {{(pid=81571) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.411721 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1498/3002] 10.222.0.23 () {58 vars in 1224 bytes} [Wed Oct 5 15:03:03 2022] DELETE /identity/v3/users/225173f51c3b46f7ae9891c71b1d461c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:03:03.416132 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27513-01d4-4c2a-a696-a1baadf87b81 None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:03.416641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27513-01d4-4c2a-a696-a1baadf87b81 None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:03.416641 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27513-01d4-4c2a-a696-a1baadf87b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:03.436027 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-6ba27513-01d4-4c2a-a696-a1baadf87b81 None None] MFA Rules not processed for user `e92af64dbb734bd49c69c6f84550ccd3`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:03:03.437247 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ba27513-01d4-4c2a-a696-a1baadf87b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:03:03.499569 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1505/3003] 10.222.0.23 () {58 vars in 958 bytes} [Wed Oct 5 15:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:03:03.509769 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:03.511793 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:03:03.568663 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505124c8-65c8-4618-8616-33f8d3aca183 admin admin] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:03.572761 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c9b940cd9e0432daa235f82becb4d38', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:03.573369 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:03.573670 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:03.573925 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:03.583516 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:03.584937 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:03.594720 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1499/3004] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:03:03.885699 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba27513-01d4-4c2a-a696-a1baadf87b81 None None] Authenticating user token {{(pid=81570) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:03:03.887242 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-6ba27513-01d4-4c2a-a696-a1baadf87b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:03:03.899563 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba27513-01d4-4c2a-a696-a1baadf87b81 None None] Validating token access rules against request {{(pid=81570) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:03:03.904948 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e92af64dbb734bd49c69c6f84550ccd3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8abb8ea74a4d492daea0a4d1c426e1f2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81570) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:03:03.905591 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:03:03.905898 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:03:03.906164 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] PATH_INFO: `/v3/projects/7a72dc2bf186400384b99db491eef760` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:03:03.907371 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a72dc2bf186400384b99db491eef760)` {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:03:03.909332 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] RBAC: Authorization granted {{(pid=81570) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:03:03.956690 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a72dc2bf186400384b99db491eef760', 'request_id': 'req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.962063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a72dc2bf186400384b99db491eef760', 'request_id': 'req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.968466 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a72dc2bf186400384b99db491eef760', 'request_id': 'req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34'} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.985063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] The token cache is being invalidate because project 7a72dc2bf186400384b99db491eef760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81570) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 05 15:03:03.985063 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.985626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.notifications [None req-982fd9ed-4f33-4115-b72b-1bc0ffed3c34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81570) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 05 15:03:03.987158 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1506/3005] 10.222.0.23 () {58 vars in 1233 bytes} [Wed Oct 5 15:03:03 2022] DELETE /identity/v3/projects/7a72dc2bf186400384b99db491eef760 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 05 15:03:13.590765 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=29,keystone:DELETE=1 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:03:13.981229 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG dbcounter [-] [81570] Writing DB stats keystone:SELECT=68,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81570) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:06:08.257697 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:06:08.259368 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:06:08.359672 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d7c58-5ccc-4447-a24b-b5fad9754d0c service neutron] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:06:08.423547 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:06:08.424161 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:06:08.424432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:06:08.424692 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:06:08.426811 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:06:08.438878 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:06:08.440677 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:06:08.451018 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1500/3006] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:06:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 05 15:06:18.446933 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG dbcounter [-] [81571] Writing DB stats keystone:SELECT=71 {{(pid=81571) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 05 15:08:09.223203 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ea547-2b00-4e68-8634-841f383e026d None None] REQUEST_METHOD: `POST` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:08:09.223809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ea547-2b00-4e68-8634-841f383e026d None None] SCRIPT_NAME: `/identity` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:08:09.223809 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89ea547-2b00-4e68-8634-841f383e026d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81570) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:08:09.274004 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.auth.core [None req-a89ea547-2b00-4e68-8634-841f383e026d None None] MFA Rules not processed for user `5b9792d95cad4f0d8708df0a608f30ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81570) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 05 15:08:09.275217 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: DEBUG keystone.common.fernet_utils [None req-a89ea547-2b00-4e68-8634-841f383e026d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81570) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:08:09.284626 dv-cinder-835878-4-iscsi devstack@keystone.service[81570]: [pid: 81570|app: 0|req: 1507/3007] 10.222.0.44 () {58 vars in 1015 bytes} [Wed Oct 5 15:08:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 05 15:08:09.300388 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] Authenticating user token {{(pid=81571) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 05 15:08:09.302051 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.fernet_utils [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81571) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 05 15:08:09.315617 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a80f0-3708-4d39-b6a8-a50b7be77db0 service placement] Validating token access rules against request {{(pid=81571) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 05 15:08:09.321517 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27292db-38f6-4672-9579-ced1904049ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5b9792d95cad4f0d8708df0a608f30ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'badaa5d0952a4dfaa32a5f179213b689', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81571) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 05 15:08:09.322154 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27292db-38f6-4672-9579-ced1904049ef service placement] REQUEST_METHOD: `GET` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 05 15:08:09.322432 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27292db-38f6-4672-9579-ced1904049ef service placement] SCRIPT_NAME: `/identity` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 05 15:08:09.322700 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27292db-38f6-4672-9579-ced1904049ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81571) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 05 15:08:09.332125 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27292db-38f6-4672-9579-ced1904049ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 05 15:08:09.333739 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27292db-38f6-4672-9579-ced1904049ef service placement] RBAC: Authorization granted {{(pid=81571) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 05 15:08:09.343991 dv-cinder-835878-4-iscsi devstack@keystone.service[81571]: [pid: 81571|app: 0|req: 1501/3008] 10.222.0.23 () {60 vars in 1365 bytes} [Wed Oct 5 15:08:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)